Display control device, image-capturing device, display control device operation method, and operation program

Abstract

A processor of a display control device according to the present invention: acquires an image including a subject; detects from the image at least part of the subject as an object of enlargement, in addition to a subject region representing the subject; inserts in a display screen an enlarged region obtained by enlarging an object region representing the object of enlargement in the display screen for displaying the image; and in a case in which an overlapping region in which overlapping of the subject region and the enlarged region occurs in the display screen, and also a condition is satisfied, executes overlap degree reduction processing for reducing a degree of overlap of the overlapping region.

Description

Display control device, imaging device, and operation method and operation program for display control device

[0001] The technology of the present disclosure relates to a display control device, an imaging device, and an operation method and operation program for the display control device.

[0002] Japanese Patent Application Laid-Open Publication No. 2010-226496 describes an imaging device that includes an imaging unit that generates first image data from an optical image of a subject within an imaging range, an autofocus unit that automatically adjusts the focus position of the imaging unit to the subject, a focus area identification unit that identifies a first area from the first image data, which is a portion of the first image data and includes the focus position, an image enlargement unit that enlarges second image data in the first image data that corresponds to the first area, a combination area setting unit that sets a second area into which the enlarged second image data is combined with the first image data so that it does not overlap with the focus position, an image combination unit that combines the enlarged second image data with the second area in the first image data, and an image output unit that outputs the first image data combined with the enlarged second image data.

[0003] Japanese Patent Application Laid-Open No. 2009-089051 describes an imaging device that includes an imaging means that images a subject and generates image data, a recording means that records the image data generated by the imaging means, a display means that displays an image based on the image data generated by the imaging means on a main screen and an image based on the image data recorded in the recording means on a sub-screen, a face detection means that detects a facial image based on the image data generated by the imaging means, an overlap determination means that, when a facial image is detected by the face detection means, determines whether the detection area of ​​the detected facial image overlaps with the area of ​​the sub-screen, and a display control means that, when the overlap determination means determines that the detection area of ​​the facial image overlaps with the area of ​​the sub-screen, controls the display of the main screen or the sub-screen so that the facial image is visible.

[0004] One embodiment of the technique of the present disclosure provides a display control device, an imaging device, and an operation method and operation program for a display control device that make it easier to check both the composition and the focus state compared to conventional methods.

[0005] In order to achieve the above object, the display control device according to the technology of the present disclosure is a display control device equipped with a processor, which acquires an image including a subject, detects from the image at least a part of the subject as an enlargement target in addition to a subject area representing the subject, inserts an enlarged area into the display screen on which the image is displayed, which is an enlarged area of ​​the target area representing the enlargement target, and if an overlapping area occurs on the display screen where the subject area and the enlarged area overlap and a condition is satisfied, performs an overlap reduction process to reduce the overlapping degree of the overlapping area.

[0006] Preferably, the processor performs the overlap reduction process by adjusting at least one of the display size and the insertion position of the magnification region.

[0007] In the overlapping region, it is preferable that the enlarged region is displayed in front of the subject region.

[0008] The condition is preferably defined by the magnitude relationship between a numerical index indicating the degree of overlap and a threshold value.

[0009] The numerical index preferably includes any one of the ratio of the area of ​​the overlapping region to the area of ​​the subject region or the area of ​​the enlarged region, the number of overlapping regions, and the distance between the subject region and the enlarged region.

[0010] It is preferable that the threshold value be changeable depending on the part of the subject in the overlapping region.

[0011] The numerical index is preferably the ratio of the area of ​​the overlap region to the area of ​​the subject region or the area of ​​the enlarged region.

[0012] When the subject region is detected as a rectangular region including the subject, the processor preferably determines at least one of whether an overlapping region exists and whether a condition is satisfied based on the overlap between the rectangular region and the enlarged region.

[0013] The target region is also preferably detected as a rectangular region containing the target to be enlarged.

[0014] When an image contains multiple subjects and at least one of the subjects is to be enlarged and inserted as an enlarged area, it is preferable that the processor determine whether or not there is an overlapping area based on the relationship between the enlarged area and the subject area corresponding to the enlarged area.

[0015] When an image includes a first subject and a second subject as subjects, and a first enlarged area that is an enlarged area corresponding to the enlarged target of the first subject and a second enlarged area that is an enlarged area corresponding to the enlarged target of the second subject are inserted, it is preferable that the processor determines the positional relationship between the first enlarged area and the second enlarged area based on the positional relationship between the first subject area representing the first subject and the second subject area representing the second subject in the image.

[0016] When multiple initial positions are set with priorities assigned as insertion positions for the enlarged area, it is preferable that the processor determines the insertion position according to the priority if each of the multiple initial positions satisfies the conditions.

[0017] When the subject is a living organism, the part detected as the enlargement target is preferably one of the subject's eyes, face, or head.

[0018] The cycle of determining whether or not to perform the overlap reduction process preferably corresponds to either the refresh rate of the display screen or the frame rate of the image.

[0019] Even if the conditions are met, it is preferable that the processor not execute the overlap reduction process if the movement amount and reduction rate of the enlarged area determined in the overlap reduction process are equal to or less than preset values.

[0020] When the processor repeatedly detects an object to be enlarged from an image, if an index relating to the accuracy of the detection of the object to be enlarged is below a predetermined standard, it is preferable that the processor performs at least one of determining the object to be enlarged and determining the content of the overlap reduction process based on past detection results.

[0021] The overlap reduction process preferably includes a process of adjusting the transparency of the overlapping regions.

[0022] Preferably, the processor determines the portion of the subject to be enlarged depending on the size of the subject within the image.

[0023] Preferably, the processor is capable of adjusting the visibility of the magnification region separately from the subject region.

[0024] An imaging device according to the technique of the present disclosure is an imaging device including any of the display control devices described above, in which the processor starts displaying the enlarged area when an imaging operation is started.

[0025] The imaging operation is preferably a focusing operation.

[0026] Preferably, the processor terminates the display of the enlarged area when the imaging operation is completed.

[0027] Preferably, the processor starts or ends the display of the enlarged area based on the operation of the release button.

[0028] A method of operating a display control device according to the disclosed technology is a method of operating a display control device equipped with a processor, in which the processor acquires an image including a subject, detects from the image a subject area representing the subject, and in addition to that, detects at least a part of the subject as an object to be enlarged, and inserts an enlarged area into the display screen on which the image is displayed, which is an enlarged object area representing the object to be enlarged, and if an overlapping area occurs on the display screen where the subject area and the enlarged area overlap and a condition is satisfied, performs an overlap reduction process to reduce the overlapping degree of the overlapping area.

[0029] The operating program of a display control device relating to the technology of the present disclosure is an operating program of a display control device equipped with a processor, which causes the processor to perform processes including acquiring an image including a subject, detecting from the image a subject area representing the subject as well as at least a part of the subject as an object to be enlarged, inserting an enlarged area into the display screen on which the image is displayed, the enlarged area being an enlarged object area representing the object to be enlarged, and, if an overlapping area occurs on the display screen where the subject area and the enlarged area overlap and conditions are met, performing an overlap reduction process to reduce the overlapping degree of the overlapping area.

[0030] According to the technology of the present disclosure, it is easier to check both the composition and the focus state compared to the prior art.

[0031] 1 is a diagram illustrating the appearance of an imaging device. FIG. 1 is a diagram illustrating an example of the configuration of an imaging device. FIG. 2 is a block diagram illustrating an example of the functional configuration of a processor. FIG. 2 is a diagram illustrating an example of eye detection as subject detection. FIG. 3 is a diagram illustrating an example of face detection as subject detection. FIG. 4 is a diagram illustrating an example of head detection as subject detection. FIG. 5 is a diagram illustrating an example of PIP display. FIG. 6 is a diagram illustrating an example of overlapping reduction processing. FIG. 7 is a diagram illustrating another example of overlapping reduction processing. FIG. 8 is a diagram illustrating an example of control information related to overlapping reduction. A flowchart illustrating an example of a processing procedure for PIP display during a focusing operation. FIG. 9 is a diagram illustrating an example of control information for Modification 1. FIG. 10 is a diagram illustrating an example of a display state in which a threshold is changed depending on a body part in Modification 1. A flowchart illustrating a processing procedure for Modification 1. FIG. 11 is a diagram illustrating an example of control information for Modification 2. FIG. 12 is a diagram illustrating an example of a display state in which a threshold is changed depending on the area of ​​a subject region in Modification 2. FIG. 13 is a diagram illustrating another example of control information for Modification 2. FIG. 14 is a diagram illustrating various indices of overlapping degree. FIG. 15 is a diagram illustrating an example in which the number of overlapping regions is used as an index of overlapping degree. FIG. 16 is a diagram illustrating an example in which the distance between a subject region and an enlarged region is used as an index of overlapping degree. FIG. 17 is a diagram illustrating an example of a method for calculating overlapping regions. FIG. 18 is a diagram illustrating an example in which the initial insertion position of an enlarged region is determined based on priority. FIG. 19 is a diagram illustrating an example of a method for determining overlapping regions when there are multiple subjects. FIG. 20 is a diagram illustrating an example in which the positional relationships between multiple subjects and multiple enlarged regions correspond to each other. FIG. 1 is a diagram showing an example of a countermeasure against hunting using a dead zone; FIG. 2 is a diagram showing an example of a countermeasure against hunting caused by detection accuracy; FIG. 3 is a diagram showing an example of determining a part to be enlarged depending on the size of a subject; FIG. 4 is a diagram showing an example of improving the visibility of an enlarged area; and FIG. 5 is a diagram showing an example of adjusting the transparency of an overlapping area as an overlap reduction process.

[0032] An example of an embodiment of the technology of the present disclosure will be described with reference to the accompanying drawings.

[0033] First, the terms used in the following description will be explained.

[0034] In the following description, "IC" is an abbreviation for "Integrated Circuit." "CPU" is an abbreviation for "Central Processing Unit." "ROM" is an abbreviation for "Read Only Memory." "RAM" is an abbreviation for "Random Access Memory." "CMOS" is an abbreviation for "Complementary Metal Oxide Semiconductor."

[0035] "FPGA" is an abbreviation for "Field Programmable Gate Array." "PLD" is an abbreviation for "Programmable Logic Device." "ASIC" is an abbreviation for "Application Specific Integrated Circuit." "OVF" is an abbreviation for "Optical View Finder." "EVF" is an abbreviation for "Electronic View Finder." "AF" is an abbreviation for "Auto Focus." "PIP" is an abbreviation for "Picture-in-Picture."

[0036] The technology of the present disclosure will be described using an interchangeable lens digital camera as an example of an embodiment of an imaging device. Note that the technology of the present disclosure is not limited to interchangeable lens digital cameras, but can also be applied to digital cameras with integrated lenses. It can also be applied to digital cameras built into smart devices, etc.

[0037] Fig. 1 is an external view of an imaging device 10, and Fig. 2 shows an example of the internal configuration of the imaging device 10. As shown in Figs. 1 and 2, the imaging device 10 is an interchangeable lens digital camera. The imaging device 10 is composed of a main body 11 and an imaging lens 12 that is interchangeably attached to the main body 11. The imaging lens 12 is attached to the front side of the main body 11 via a camera-side mount 11A and a lens-side mount 12A.

[0038] The main body 11 is provided with operation units such as a dial 24, a release button 22, and a display 15 with a touch panel function. These operation units constitute an operation device 13 that accepts operations by a user. Operation modes of the imaging device 10 include, for example, a still image capture mode, a video capture mode, and an image display mode. Furthermore, the still image capture mode includes a continuous shooting mode. For example, the dial 24 is operated by the user when setting the operation mode. Furthermore, the release button 22 is operated by the user when starting still image capture or video capture. Furthermore, the display 15 with a touch panel function is used to display various setting screens as well as to play back and display captured images. Furthermore, the display 15 with a touch panel function is operated by the user when specifying an AF area to be focused on from within the imaging area.

[0039] The main body 11 is also provided with a viewfinder 14. The viewfinder 14 is a hybrid viewfinder (registered trademark). A hybrid viewfinder is a viewfinder that selectively uses, for example, an optical viewfinder (hereinafter referred to as "OVF") and an electronic viewfinder (hereinafter referred to as "EVF"). The user can observe an optical image or a live view image of the subject displayed by the viewfinder 14 through the viewfinder eyepiece.

[0040] The display 15 is provided on the rear side of the main body 11. Instead of using the viewfinder 14, the user can also observe a live view image displayed on the display 15.

[0041] The main body 11 and the imaging lens 12 are electrically connected by electrical contacts 11B provided on the camera-side mount 11A coming into contact with electrical contacts 12B provided on the lens-side mount 12A.

[0042] The imaging lens 12 includes an objective lens 30, a focus lens 31, a rear end lens 32, and an aperture 33. The components are arranged along the optical axis A of the imaging lens 12 in the following order from the objective side: objective lens 30, aperture 33, focus lens 31, and rear end lens 32. The objective lens 30, focus lens 31, and rear end lens 32 constitute an imaging optical system. The type, number, and arrangement order of the lenses that make up the imaging optical system are not limited to the example shown in FIG. 2 .

[0043] The imaging lens 12 also has a lens driver 34. The lens driver 34 is configured with, for example, a CPU, RAM, and ROM. The lens driver 34 is electrically connected to a processor 40 in the main body 11 via electrical contacts 12B and 11B.

[0044] The lens driving unit 34 drives the focus lens 31 and the diaphragm 33 based on a control signal transmitted from the processor 40. The lens driving unit 34 controls the driving of the focus lens 31 based on a control signal for focus control transmitted from the processor 40 in order to adjust the focus position of the imaging lens 12. The processor 40 performs focus position detection using a phase difference method, for example.

[0045] The diaphragm 33 has an aperture whose diameter is variable around the optical axis A. The lens driver 34 controls the drive of the diaphragm 33 based on an aperture adjustment control signal sent from the processor 40 in order to adjust the amount of light incident on the light receiving surface 20A of the image sensor 20.

[0046] The main body 11 also contains an image sensor 20, a processor 40, and a memory 42. The operations of the image sensor 20, the memory 42, the operation device 13, the viewfinder 14, and the display 15 are controlled by the processor 40.

[0047] The processor 40 is configured by, for example, a CPU. In this case, the processor 40 executes various processes based on a program 43 stored in the memory 42. The processor 40 may be configured by a collection of multiple IC chips. The memory 42 is configured by at least one of various storages such as a RAM, a flash memory, a hard disk drive, etc. The memory 42 may also include a ROM.

[0048] The imaging sensor 20 is, for example, a CMOS image sensor. The imaging sensor 20 is arranged so that its optical axis A is perpendicular to the light-receiving surface 20A and is located at the center of the light-receiving surface 20A. Light that has passed through the imaging lens 12 is incident on the light-receiving surface 20A. A plurality of pixels that generate signals by performing photoelectric conversion are formed on the light-receiving surface 20A. The imaging sensor 20 photoelectrically converts the light incident on each pixel to generate and output an image signal D. The imaging sensor 20 is an example of an "imaging element" according to the technology of the present disclosure.

[0049] Furthermore, as an example, a Bayer color filter array is arranged on the light receiving surface 20A of the image sensor 20, and a color filter of either R (red), G (green), or B (blue) is arranged opposite each pixel.

[0050] The imaging device 10 employs a phase difference focusing method, as an example. As is well known, the phase difference focusing method uses a pair of phase difference detection pixels arranged with parallax and having different incident light beams due to pupil division. In the phase difference focusing method, the pair of phase difference detection pixels detects the amount of deviation of the focus lens 31 from the in-focus position as a phase difference, and the focus lens 31 is moved to the in-focus position based on the detected phase difference. The imaging device 10 employs an image plane phase difference focusing method, and the phase difference detection pixels are provided on at least a portion of the multiple pixels arranged on the light receiving surface 20A of the imaging sensor 20. Multiple pairs of phase difference detection pixels are dispersedly arranged within the light receiving surface 20A, allowing the imaging device 10 to set an AF area across the entire imaging range captured by the light receiving surface 20A. Instead of the phase difference focusing method, a contrast detection focusing method may be employed, in which the focus lens 31 is moved while searching for the in-focus position based on a signal output by the imaging sensor 20.

[0051] 3 shows an example of the functional configuration of the processor 40. The processor 40 executes processing in accordance with a program 43 stored in a memory 42 to realize various functional units. As shown in FIG. 3, for example, the processor 40 realizes a main control unit 50, an imaging control unit 51, an image processing unit 52, a display control unit 53, an AF control unit 55, and a subject detection unit 64. The program 43 is an example of an "operation program" according to the technology of the present disclosure.

[0052] The main control unit 50 performs overall control of the operation of the imaging device 10 based on output information from the operation device 13. The imaging control unit 51 controls the imaging sensor 20 to perform imaging processing that causes the imaging sensor 20 to perform an imaging operation. The imaging control unit 51 drives the imaging sensor 20 in a still image imaging mode or a video imaging mode.

[0053] The image sensor 20 outputs an image signal D including an image signal and a signal from the phase difference detection pixel.

[0054] The image processing unit 52 acquires the image signal D output from the image sensor 20 and performs image processing such as demosaic processing on the acquired image signal D.

[0055] The AF control unit 55 performs focusing control by adjusting the focus lens 31 to a focusing position. The AF control unit 55 is made up of an AF area setting unit 54 and an AF calculation unit 57.

[0056] The AF area setting unit 54 sets an AF area RA (see FIG. 7, etc.) that is an area to be focused within the imaging area 20B. For example, as shown in FIG. 7, the AF area setting unit 54 sets an area that includes at least a part of the subject detected as the focus target by the subject detection unit 64 as the AF area RA. In FIG. 7, an example is shown in which the subject is a person, the eyes are detected as the focus target, and the area that includes the detected eyes is set as the AF area RA.

[0057] Returning to FIG. 3 , the subject detection unit 64 recognizes the subject based on the image signal D using image recognition technology based on a pattern matching technique or an AI (Artificial Intelligence) technique. The subject to be recognized is at least a part of the subject, that is, both the entire subject and a part of the subject. Examples of subjects include people, animals, and vehicles. If the subject is a person or an animal, the part of the subject is the eyes, face, head, etc. Examples of vehicles include automobiles, trains, and airplanes. If the subject is a vehicle, the part of the subject is the front, windows, rear, etc.

[0058] The subject detection unit 64 can continuously detect the subject and parts thereof, for example, when the user is framing to check the composition while pressing the release button 22 halfway, or when the user is performing continuous shooting by taking multiple images while pressing the release button 22 all the way down. This makes it possible to track the AF area RA even if the subject moves. The subject detection unit 64 continuously outputs information about the AF area RA that moves in accordance with the movement of the subject to the AF area setting unit 54.

[0059] The subject detection unit 64 detects an area representing the entire subject as a subject area SA (see, for example, FIG. 6 ), and an area representing a portion of the subject as a target area PA (see, for example, FIG. 6 ). The target area PA is an area set as a focus target, and also an area including an enlargement target SP that is enlarged in PIP display, which will be described later. The subject detection unit 64 outputs information about the subject area SA and the target area PA to the display control unit 53 and the AF area setting unit 54.

[0060] The AF area setting unit 54 can also set an area specified by the user via the operation device 13 as the AF area RA. For example, the AF area RA can be specified by touching the touch panel of the display 15, which serves as the operation device 13, with a finger.

[0061] The AF calculation unit 57 acquires information about the AF area RA from the AF area setting unit 54, and calculates the defocus amount within the AF area RA based on the signals of the phase difference detection pixels in the image signal D. The defocus amount represents the amount of deviation from the in-focus position of the focus lens 31, and the main control unit 50 adjusts the in-focus position by driving the focus lens 31 via the lens driving unit 34 based on the defocus amount. As a result, the subject within the AF area RA becomes in-focus.

[0062] The display control unit 53 causes the finder 14 or the like to display an image represented by the image signal D that has been subjected to image processing by the image processing unit 52. Furthermore, the display control unit 53 causes the finder 14 or the like to display a live view image based on the image signal D that is periodically input from the image processing unit 52 during an imaging preparation operation before still image capture or video capture.

[0063] When performing live view display in which a live view image is displayed on a display screen such as the viewfinder 14, the imaging device 10 has a PIP function that enlarges a target area PA set as an AF area RA and inserts the enlarged enlarged area LPA (see FIG. 7, etc.) into the display screen as a sub-screen. The display control unit 53 has a PIP processing unit 61 that performs PIP processing.

[0064] Specific examples of subject detection, which is a prerequisite for PIP processing, will be described with reference to FIGS. 4 to 6. All of FIGS. 4 to 6 show examples in which the subject is a person. The examples in FIGS. 4 to 6 show examples in which a person subject S and a portion of the subject S are detected as an enlargement target SP. FIG. 4 shows an example in which an eye SP(E) is detected as the enlargement target SP. The subject detection unit 64 detects the subject S and the eye SP(E) from the image 36 represented by the image signal D using image recognition processing, and detects a subject area SA representing the subject S and a target area PA(E) representing the eye SP(E). The subject area SA and the target area PA(E) are detected as rectangular areas including the subject S or the eye SP(E), respectively.

[0065] 5 shows an example of detecting a face SP(F) as the enlargement target SP. The subject detection unit 64 detects the subject S and the face SP(F) from the image 36 represented by the image signal D through image recognition processing, and detects a subject area SA representing the subject S and a target area PA(F) representing the face SP(F). The subject area SA and the target area PA(F) are detected as rectangular areas that include the subject S or the face SP(F), respectively.

[0066] 6 shows an example in which a head SP(H) is detected as the enlargement target SP. The subject detection unit 64 detects the subject S and the head SP(H) from the image 36 represented by the image signal D through image recognition processing, and detects a subject area SA representing the subject S and a target area PA(H) representing the head SP(H). The subject area SA and the target area PA(H) are detected as rectangular areas that include the subject S or the head SP(H), respectively.

[0067] The target area PA is an area set as the AF area RA. In the PIP processing, the PIP processing unit 61 enlarges the target area PA and performs image synthesis by inserting the enlarged image as a sub-screen within the display screen that displays the entire image 36.

[0068] Fig. 7 shows an example of a PIP display using PIP processing. In the example shown in Fig. 7, an enlarged area LPA, which is an enlarged image of a target area PA(E) representing the eyes SP(E) of the subject S, is inserted as a sub-screen within a display screen displaying an image 36 including the subject S. By performing a PIP display in which the enlarged area LPA is inserted within the image 36 in this manner, it becomes possible to confirm the composition, such as where and how large the subject S is to be placed within the entire image 36, and to confirm the focus state, i.e., whether the subject S is in focus, by using the enlarged area LPA of the eyes SP(E).

[0069] The insertion position of the enlarged area LPA is initially set to, for example, the lower right corner or the lower left corner of the display screen. In the case of PIP display, if the insertion position of the enlarged area LPA is fixed, the enlarged area LPA and the subject area SA may overlap.

[0070] Returning to Fig. 3, the PIP processing unit 61 is provided with an overlapping reduction processing unit 61A. When the enlarged area LPA and the subject area SA overlap, the overlapping reduction processing unit 61A executes an overlapping reduction process to reduce the overlapping. The memory 42 stores control information 66. The control information 66 is information that defines the processing rules for executing the overlapping reduction process.

[0071] 8 and 9 show an example of the overlap reduction process. As shown in the upper diagram of each of FIGS. 8 and 9, an example is shown in which the subject area SA and the enlarged area LPA overlap as the subject S moves, as indicated by the diagram in the upper row. When an overlap area OV, where the subject area SA and the enlarged area LPA overlap, occurs on the display screen of the image 36 and a preset condition is satisfied, the overlap reduction processing unit 61A executes the overlap reduction process to reduce the overlap degree ODG (see FIG. 10 ) of the overlap area OV. The preset condition is an example of a "condition" according to the technology of the present disclosure.

[0072] The overlap reduction processing section 61A calculates the presence or absence of an overlap area OV and the area of ​​the overlap area OV based on the coordinate information of the subject area SA and the enlarged area LPA.

[0073] The enlarged area LPA is an area for checking the focus state, and therefore in this example, in the overlap area OV, it is displayed in front of the subject area SA.

[0074] As an example of the overlap reduction process, the overlap reduction processor 61A executes a process for adjusting the insertion position of the enlarged area LPA, as shown in Fig. 8 . Fig. 8 shows an example in which the insertion position of the enlarged area LPA, which was inserted in the lower right corner based on the viewing side of the image 36, is changed to the lower left corner. Based on the position of the subject area SA in the image 36, the overlap reduction processor 61A searches for a position where the subject area SA and the enlarged area LPA do not overlap or overlap minimally, and determines the searched position as the insertion position. The enlarged area LPA is then moved to the determined insertion position.

[0075] The overlap reduction processing unit 61A also executes processing to adjust the display size of the enlarged area LPA as shown in Fig. 9 as an example of the overlap reduction processing. Fig. 9 shows an example in which the display size of the enlarged area LPA is reduced. The overlap reduction processing unit 61A calculates the area of ​​the area near the current insertion position of the enlarged area LPA based on the position of the subject area SA within the image 36, and determines a reduction rate for the enlarged area LPA that does not overlap with the subject area SA. The enlarged area LPA is then reduced at the determined reduction rate.

[0076] In the examples shown in Figures 8 and 9, the overlap is completely eliminated by the overlap reduction process, eliminating the overlap area OV. However, in reality, the overlap may not be completely eliminated, and the overlap area OV may remain. The overlap reduction process is sufficient as long as the overlap ODG is reduced by comparing the pre-processing and post-processing results, and a portion of the overlap area OV may remain. The overlap ODG is an index indicating the degree of overlap between the subject area SA and the enlarged area LPA. For example, the index is a numerical index such as the ratio of the area of ​​the overlap area OV to the area of ​​the subject area SA. The area is calculated using the number of pixels and coordinate information defining the pixel position in the image 36. When the subject area SA and the enlarged area LPA are detected as rectangular areas, the area where the rectangular areas overlap is the overlap area OV, as shown as an example in Figures 8 and 9.

[0077] As shown in Fig. 10 , the control information 66 includes information such as a condition for determining whether or not to perform the overlap reduction process, the initial position of the enlarged area LPA, and a determination cycle. In the example shown in Fig. 10 , the condition is, for example, a condition that the overlap degree ODG is equal to or greater than a predetermined threshold value TH (ODG ≥ TH). In the example of Fig. 10 , the overlap degree ODG is defined as the ratio of the area of ​​the overlap region OV to the area of ​​the subject area SA. In this way, the condition is defined, for example, by the magnitude relationship between a numerical index indicating the degree of overlap of the overlap region OV, such as the overlap degree ODG, and the threshold value TH.

[0078] The initial position is set to the lower left or lower right corner of the display screen. Furthermore, in the example shown in FIG. 10 , priorities are assigned to the initial positions, with the lower right corner being the first priority and the lower left corner being the second priority. For example, if the insertion position of the enlarged area LPA is set to the lower right corner, which has the first priority, and overlap with the subject area SA occurs, the lower left corner, which has the second priority, is selected. The determination period is a period for determining whether or not to perform the overlap reduction process, and more specifically, a period for determining whether or not a condition is satisfied. In the example shown in FIG. 10 , the determination period is defined as the frame rate, which is the imaging period of the image sensor 20, or the refresh rate, which is the update period of the display screen of the display 15. Either of the determination periods can be selected by user settings. A device including a processor 40 including a display control unit 53 and a memory 42 is an example of a "display control device" according to the technology of the present disclosure, and the imaging device 10 is an example of an "imaging device" according to the technology of the present disclosure.

[0079] The operation of the above configuration will be described with reference to the flowchart shown in Fig. 11. The flowchart shown in Fig. 11 shows the operation procedure of display control during the focusing operation of the imaging device 10. For example, when a still image imaging mode is selected as the operating mode, the display control unit 53 starts a live view display, for example, in the viewfinder 14, in step S1100. The user can check the composition through the live view display.

[0080] In step S1200, the display control unit 53 waits for an instruction to start a focusing operation. When a focusing operation is instructed by half-pressing the release button 22 (Y in step S1200), the subject detection unit 64 detects the subject S and the object to be enlarged SP from the image 36. As shown in FIG. 7, when the subject S is a person, the object to be enlarged SP is, for example, the eyes SP(E) of the subject S. In step S1300, when the subject S and the object to be enlarged SP are detected (Y in step S1300), the process proceeds to step S1400, and the PIP processing unit 61 in the display control unit 53 starts PIP display.

[0081] On the other hand, the AF control unit 55 sets the target area PA including the enlargement target SP as the AF area RA, and performs a focusing operation.

[0082] In step S1500, the PIP processing unit 61 enlarges the target area PA representing the detected enlargement target SP, and inserts the enlarged enlarged area LPA into the initial position of the display screen displaying the image 36. As a result, the enlarged area LPA is displayed as shown in Fig. 7, and the user can check whether the subject S is in focus, i.e., the in-focus state of the subject S, while observing the enlarged area LPA.

[0083] In step S1600, the PIP processing unit 61 monitors whether or not an overlapping area OV where the subject area SA and the enlarged area LPA overlap is present on the display screen.

[0084] In step S1600, if it is determined that an overlapping area OV exists (Y in step S1600), the PIP processing unit 61 determines whether or not the conditions regarding the overlapping area OV shown in the control information 66 in FIG. 10 are satisfied.

[0085] If the condition is met (Y in step S1700), the process proceeds to step S1800, where the overlap reduction processing unit 61A of the PIP processing unit 61 executes overlap reduction processing to adjust the insertion position or display size of the enlarged area LPA, as shown in Figures 8 and 9. This reduces the overlap ODG between the subject area SA and the enlarged area LPA. Therefore, the imaging device 10 makes it easier to check both the composition and the focus state compared to when the overlap reduction processing is not executed.

[0086] In step S1900, the PIP processor 61 waits for the completion of the focusing operation. The focusing operation ends, for example, when the half-pressed release button 22 is fully pressed or when the half-press is canceled. The PIP processor 61 repeats the processes of steps S1300 to S1800 while the focusing operation continues (N in step S1900). The subject S and the enlargement target SP are detected in step S1300, and this detection is performed in accordance with the frame rate of the image sensor 20, that is, each time a live view image is acquired. The PIP display in step S1400 is updated each time the subject S and the enlargement target SP are detected.

[0087] On the other hand, if the focusing operation is completed in step S1900 (Y in step S1900), the process proceeds to step S2000, where the PIP processing unit 61 ends the PIP display. If the PIP display is completed, the display control unit 53 proceeds to step S2100, where the above process is repeated until the live view display is completed.

[0088] As described above, the technology disclosed herein detects, from the image 36, a subject area SA representing the subject S, as well as at least a portion of the subject S as an enlargement target SP, and inserts an enlarged area LPA, which is an enlarged version of the target area PA representing the enlargement target SP, into the display screen displaying the image 36. If an overlap area OV, in which the subject area SA and the enlargement area LPA overlap, occurs on the display screen and satisfies a condition defined by the relationship between the overlap degree ODG and a threshold value TH, an overlap reduction process is executed to reduce the overlap degree ODG of the overlap area OV. Therefore, the technology disclosed herein makes it easier to confirm both the composition and the focus state compared to conventional techniques.

[0089] Furthermore, since the processor 40 performs the overlap reduction process by adjusting at least one of the display size and the insertion position of the enlarged area LPA, the process may be simpler than other methods.

[0090] Furthermore, in the overlapping area OV, the enlarged area LPA is displayed in front of the subject area SA, so it is easier to check the focus state compared to when the subject area SA is displayed in front.

[0091] In addition, since the condition for determining whether to perform the redundancy reduction process is determined by the magnitude relationship between the numerical index indicating the redundancy ODG and the threshold value TH, the process may be simpler than if a numerical index is not used.

[0092] Furthermore, the numerical index is the ratio of the area of ​​the overlapping area OV to the area of ​​the subject area SA, and is therefore intuitively easy to understand.

[0093] Furthermore, when the subject area SA is detected as a rectangular area including the subject S, the processor 40 determines at least one of whether an overlap area OV exists and whether a condition for determining whether to perform an overlap reduction process (an example of a first condition) is met, based on the overlap between the rectangular area and the enlarged area LPA. When the determination is not based on a rectangular area, the processing becomes less complicated compared to, for example, when the outline of the subject S is extracted and the area within the extracted outline is used. Furthermore, because the target area PA is also detected as a rectangular area including the target area PA, the processing becomes less complicated for the same reason.

[0094] Furthermore, when the subject S is a living organism, the part detected as the enlargement target SP is any one of the eyes, face, and head of the subject S. When the subject S is a living organism, the focus state of the eyes, face, etc. is often important, so it is possible to provide a display that meets the needs of the user.

[0095] Furthermore, the cycle for determining whether or not to perform the redundancy reduction process corresponds to either the refresh rate of the display screen or the frame rate of the images 36 captured by the image sensor 20. Therefore, the real-time nature of the redundancy reduction is improved compared to when the cycle is longer than the refresh rate or the frame rate.

[0096] Furthermore, the processor 40 of the imaging device 10 starts the PIP display and starts displaying the enlarged area LPA when the focusing operation is started. Furthermore, when the focusing operation is completed, the processor 40 ends the PIP display and ends displaying the enlarged area LPA. Since the display of the enlarged area LPA is timed to coincide with the start or end of the focusing operation, it is easy to check the focus state. Furthermore, the processor 40 starts or ends displaying the enlarged area LPA based on operation of the release button 22. Since the focusing operation is often performed in response to operation of the release button 22, linking the display of the enlarged area LPA to operation of the release button 22 provides high convenience.

[0097] The enlarged area LPA may be displayed not only during the focusing operation but also during other imaging operations. For example, when a live view display is performed in a video shooting mode, a PIP display may be performed to display the enlarged area LPA. The live view display including the enlarged area LPA may be performed both during the video shooting standby state and during video recording.

[0098] In the above embodiment, the display control unit 53 displays a live view on the viewfinder 14 and performs PIP processing on the live view display, but it may also display a live view on the display 15 instead of or together with the viewfinder 14.

[0099] Various Modifications The technology of the present disclosure is not limited to the above-described embodiment, and various modifications are possible as described below.

[0100] (Variation 1: Changing the Threshold Depending on the Part of the Subject in the Overlap Region) In the above embodiment, an example has been described in which the threshold TH is uniformly determined as a condition for whether or not to perform the overlap reduction process, as shown in FIG. 10 . However, as shown in FIG. 12 , the threshold TH may be changeable depending on the part of the subject S in the overlap region OV. The example shown in FIG. 12 is an example in which two conditions are set: a condition (F) for when the part of the subject S in the overlap region OV is the face, and a condition (B) for when the part of the subject S is the body. The overlap degree ODG(F) is the ratio of the area of ​​the overlap region OV to the area of ​​the face of the subject S, and the overlap degree ODG(B) is the ratio of the area of ​​the overlap region OV to the area of ​​the body of the subject S. As for the threshold TH, a threshold TH(B) for the body and a threshold TH(F) for the face are respectively set.

[0101] 12, the body threshold TH(B) is larger than the face threshold TH(F). Therefore, when comparing simply the area ratios, if the body overlapping degree ODG(B) is not larger than the face overlapping degree ODG(F), the overlapping degree reduction process will not be executed.

[0102] This will be explained using the specific example shown in Fig. 13. The diagram in the upper part of Fig. 13 is an example in which an enlarged area LPA, in which the enlargement target SP is the eye SP(E), overlaps with a subject area SA showing a close-up of the face, and the overlapping area OV includes the face SP(F) of the subject S. On the other hand, the diagram in the lower part of Fig. 13 is an example in which the same enlarged area LPA overlaps with a subject area SA showing almost the entire body of the subject S, and the overlapping area OV includes parts of the body. In the examples and , the area of ​​the overlapping area OV is the same.

[0103] 12, when the part of the subject S in the overlapping region OV is the face SP(F), the threshold value TH(F) is smaller than the threshold value TH(B) when the part of the subject S in the overlapping region OV is the body. Therefore, even if the area of ​​the overlapping region OV is the same, for example, if the overlapping region OV is a face, the overlapping reduction process is executed, but if the overlapping region OV is a body, the overlapping reduction process is not executed. In other words, whether or not the overlapping reduction process is executed depends on the part of the subject S.

[0104] An example of a specific processing procedure is shown in FIG. 14. FIG. 14 is a flowchart showing a more detailed procedure related to step S1700 shown in FIG. 11. In FIG. 14, the overlap reduction processing unit 61A determines in step S1710 whether a part of the subject S in the overlap region OV is a face in the condition determination related to the overlap region OV. If it is determined to be a face (Y in step S1710), the process proceeds to step S1711, where it is determined whether condition (F) corresponding to a face is satisfied. If condition (F) is satisfied, the process proceeds to step S1713, where it is determined that the condition is satisfied. If condition (F) is not satisfied, the process proceeds to step S1714, where it is determined that the condition is not satisfied.

[0105] On the other hand, if it is determined in step S1710 that the object is not a face, the redundancy reduction processing unit 61A proceeds to step S1712, where it determines whether condition (B) corresponding to a body is satisfied. As in the case of condition (F), in the case of condition (B), the process proceeds to step S1713 or step S1714 depending on the determination result of step S1712. The determination results of step S1713 and step S1714 correspond to the determination result of step S1700 shown in FIG. 11.

[0106] Although perceptions vary from user to user, for example, when checking the composition, the user's tolerance for the overlapping area OV may vary depending on the part of the subject S, such as being bothered by the overlapping area OV occurring on the face SP(F) of the subject S, but not so much by the overlapping area OV occurring on the body. As shown in Fig. 12, by making the threshold value TH variable depending on the part of the subject S in the overlapping area OV, it becomes possible to flexibly respond to such user tolerance.

[0107] In the case of this modification 1, the redundancy reduction processing unit 61A uses the detection results of the subject detection unit 64 to recognize the parts in the overlap region OV. For example, the subject detection unit 64 uses AI object recognition technology to recognize the parts of the detected subject S, such as whether it is the face or body, and outputs the detection result including the recognition result to the redundancy reduction processing unit 61A. Based on the recognition results from the subject detection unit 64, the redundancy reduction processing unit 61A can grasp information such as the parts included in the subject region SA and the parts of the enlargement target SP included in the target region PA, as well as information such as the parts in the overlap region OV.

[0108] (Variation 2: Changing the threshold value according to the area of ​​the subject region) Furthermore, as shown in Figures 15 and 16, even if the part of the subject S in the overlap region OV is the same, the threshold value TH may be changed according to the area of ​​the subject region SA representing the subject S in the image 36. The conditions shown in Figure 15 are the same as the conditions shown in Figures 10 and 12, and the overlap degree ODG is the ratio of the area of ​​the overlap region OV to the area of ​​the subject region SA. Variation 2 is an example in which the value of the threshold value TH compared with the overlap degree ODG is changeable according to the area of ​​the subject region SA. As shown in the graph in Figure 15, for example, the larger the area of ​​the subject region SA, the larger the threshold value TH.

[0109] In Figure 16, which shows a specific example, a close-up of the face of subject S is shown, and the area including the face and shoulders is the subject area SA. In the upper and lower diagrams of Figure 16, the areas of the enlarged area LPA and overlapping area OV are the same. However, the areas of the subject area SA are different, with the subject area SA being larger in the upper diagram. Here, the overlap degree ODG(F) is defined as the ratio of the area of ​​the overlapping area OV to the area of ​​the subject area SA.

[0110] In such a case, if the threshold value TH is constant, the definition of the overlapping degree ODG indicates that, for the same area of ​​the overlapping region OV (numerator), the smaller the area of ​​the subject region SA (denominator), the larger the overlapping degree ODG, and thus the condition is more likely to be met. When the condition is met, the overlapping degree reduction process is executed. Furthermore, if the overlapping degree reduction process is executed frequently, hunting, in which the insertion position of the enlarged region LPA is frequently moved or the display size is frequently changed, is likely to occur. Specifically, as shown in the lower part of Figure 16, compared to the upper part of Figure 16, the smaller the area of ​​the subject region SA, the larger the overlapping degree ODG, and the more likely the condition is met, which makes hunting more likely to occur. Because the enlarged region LPA is used to confirm the focus state, it is preferable that the size and insertion position of the enlarged region LPA not change frequently, regardless of the size of the subject region SA.

[0111] Therefore, in Modification 2, the threshold value TH is changed according to the area of ​​the subject area SA, such that the smaller the area of ​​the subject area SA, the smaller the threshold value TH is set, as shown in the graph of Fig. 15. In this way, hunting can be suppressed even when the subject area SA is small.

[0112] In the second modification, the overlap degree ODG is defined as the ratio of the area of ​​the overlap region OV to the area of ​​the subject region SA, but it may also be defined as the ratio of the area of ​​the overlap region OV to the area of ​​the enlarged region LPA, as in the overlap degree ODG2 shown in Fig. 17. Even in this case, hunting can be suppressed by making the threshold value TH variable depending on the area of ​​the subject region SA.

[0113] 16, the larger the subject area SA is, the larger the area that the subject area SA occupies in the image 36, so there is less space to insert the enlarged area LPA while avoiding the subject area SA in the case of the upper row compared to the case of the lower row . Therefore, the probability that the subject area SA will overlap with the enlarged area LPA increases. When the subject area SA is small, the area that the subject area SA occupies in the image 36 is small, so there is relatively more space to insert the enlarged area LPA while avoiding the subject area SA.

[0114] In this case, when the subject area SA is large, overlapping is likely to occur, and therefore the area of ​​the overlapping area OV that overlaps with the enlarged area LPA (which becomes the numerator of the overlapping degree ODG2) also becomes large, and the overlapping degree ODG2 is likely to be large compared to when the subject area SA is small. In particular, when considering a case where the subject S moves frequently, if the subject S is large, even a slight movement of the subject S is likely to increase the area of ​​the overlapping area OV, and as a result, hunting is likely to occur.

[0115] Therefore, even when using the overlap degree ODG2 in which the enlarged area LPA is used as the denominator, the larger the subject area SA, the higher the threshold value TH2, as shown in the graph in Fig. 17. By changing the threshold value TH2 in this way, it is possible to suppress hunting when the subject area SA is large.

[0116] (Variation 3: Variation of Overlapping Degree) As shown in Fig. 18, various conditions, such as Condition 1 to Condition 4, can be considered as conditions for whether or not to perform the overlapping degree reduction process, and any of them may be used. Condition 1 is the same as the condition shown in Figs. 10 and 12, and is a condition in which the ratio of the area of ​​the overlapping region OV to the area of ​​the subject region SA is used as the overlapping degree ODG1 as a numerical index. Condition 2, as shown in Fig. 17, is a condition in which the ratio of the area of ​​the overlapping region OV to the area of ​​the enlarged region LPA is used as the overlapping degree ODG2 as a numerical index.

[0117] Condition 3 is a condition in which the number of overlapping regions OV is used as the numerical index for the overlapping degree ODG3. As the number of overlapping regions OV decreases, the overlapping degree ODG3 decreases, and therefore the overlapping degree reduction process is performed when the overlapping degree ODG3 is equal to or greater than a threshold value TH3. Condition 4 is a condition in which the distance between the subject region SA and the enlarged region LPA is used as the numerical index for the overlapping degree ODG4. In the case of condition 4, the longer the distance, the more the overlapping degree ODG4 decreases, and therefore the overlapping degree reduction process is performed when the overlapping degree ODG4 is equal to or less than a threshold value TH4.

[0118] FIG. 19 shows a specific example of using condition 3. In the example shown in FIG. 19 , image 36 includes multiple first subjects S(1) and second subjects S(2), and an enlarged area LPA corresponding to first subject S(1) is inserted. As shown in the upper diagram of FIG. 19 , two overlapping areas OV are generated: an overlapping area OV(1) with first subject area SA(1) representing first subject S(1), and an overlapping area OV(2) with second subject area SA(2) representing second subject S(2). In this case, if threshold TH3 is "2," condition 3 is satisfied. Therefore, as shown in the lower diagram of FIG. 19 , the overlapping reduction processing unit 61A performs an overlapping reduction process to move the insertion position of enlarged area LPA, thereby reducing the number of overlapping areas OV to one.

[0119] FIG. 20 shows a specific example when condition 4 is used. In the example shown in FIG. 20, an overlapping area OV occurs between the subject area SA and the enlarged area LPA. The overlapping reduction processor 61A calculates the distance DST between the center O(S) of the subject area SA and the center O(E) of the enlarged area LPA, and uses this distance DST as the overlapping degree ODG4. When the overlapping degree ODG4 is equal to or less than a threshold value TH4, the subject area SA and the enlarged area LPA are considered to be close to each other, and the degree of overlapping is considered to be large. Therefore, when the overlapping degree ODG4 is equal to or less than the threshold value TH4, the overlapping reduction processor 61A performs the overlapping reduction process by moving the insertion position of the enlarged area LPA in a direction that increases the distance DST or by reducing the display size.

[0120] The distance DST may be the distance between the centers of gravity of the subject S included in the subject area SA and the enlargement target SP included in the enlargement area LPA.

[0121] (Variation 4: Method of Calculating Overlapping Area) In the above embodiment, the method of calculating the area of ​​the overlapping area OV is to calculate the area of ​​the overlapping area between the rectangular areas of the subject area SA and the enlarged area LPA. Instead of such overlapping rectangular areas, calculation may be performed based on the outline of the subject S, as shown in FIG. 21 . Here, calculation based on the outline means a method of calculating the overlapping area OV by obtaining the outline of the face SP(F) and the outline of the eyes SP(E) of the subject S, and calculating the area where the internal areas defined by the outlines overlap. This method makes it possible to calculate the overlapping portion more accurately than when calculating using a rectangular area.

[0122] (Variant 5: Priority of insertion position) As shown in Figure 22, when multiple initial positions are set with priorities as insertion positions for the enlarged area LPA, the PIP processing unit 61 of the processor 40 may determine the insertion position according to the priority if each of the multiple initial positions satisfies the conditions.

[0123] As shown in the control information 66 of FIG. 10 , multiple initial positions, such as the lower right corner or the lower left corner, may be set as the initial position for the insertion position of the enlarged area LPA. In this case, the PIP processing unit 61 selects, for example, the initial position with the lowest overlapping degree ODG from among the multiple initial positions. However, as shown in FIG. 22 , multiple initial positions may both satisfy the conditions for the overlapping area OV and have the same overlapping degree ODG. In such a case, the PIP processing unit 61 may determine the insertion position according to the priorities set for the multiple initial positions. As shown in the control information 66 of FIG. 10 , multiple initial positions are set with priorities. In the example of FIG. 10 , the lower right corner is ranked first and the lower left corner is ranked second. Therefore, if the overlapping degree ODG is the same for both initial positions, the PIP processing unit 61 selects the lower right corner, which has the highest priority, as the insertion position. In this way, setting priorities makes it easier to determine the final insertion position compared to when priorities are not set.

[0124] (Variation 6: Determination of Overlap Area When Multiple Subjects Are Included) As shown in FIG. 23 , when an image 36 includes multiple subjects S and at least one enlargement target SP among the multiple first subjects S(1) and second subjects S(2) is inserted as an enlargement area LPA, the PIP processing unit 61 of the processor 40 may determine the presence or absence of an overlap area OV based on the relationship between the enlargement area LPA and the subject area corresponding to the enlargement area (first subject area SA(1) in this example). In other words, the PIP processing unit 61 only considers the overlap area OV(1) with the first subject area SA(1), and does not consider the overlap area OV(2) with the second subject area SA(2) that does not correspond to the enlargement area LPA. This prevents the determination of the overlap area OV from becoming complicated.

[0125] 24 , consider a case where an image 36 includes a first subject S(1) and a second subject S(2) as subjects S, and a first enlarged area LPA(1) that is an enlarged area LPA corresponding to the eye SP(E) that is the enlargement target SP of the first subject S(1) and a second enlarged area LPA(2) that is an enlarged area LPA corresponding to the eye SP(E) that is the enlargement target SP of the second subject S(2) are inserted. In this case, the PIP processing unit 61 of the processor 40 may determine the positional relationship between the first enlarged area LPA(1) and the second enlarged area LPA(2) based on the positional relationship between the first subject area SA(1) representing the first subject S(1) and the second subject area SA(2) representing the second subject S(2) in the image 36.

[0126] As shown in the upper diagram of FIG. 24 , the positional relationship between the first subject S(1) and the second subject S(2) is such that the first subject S(1) is located on the left and the second subject S(2) is located on the right. The positional relationship between the first enlarged area LPA(1) and the second enlarged area LPA(2) also follows suit, with the first enlarged area LPA(1) inserted on the left and the second enlarged area LPA(2) inserted on the right. Consider a case in which the PIP processor 61 performs overlap reduction processing by moving the insertion positions of the first enlarged area LPA(1) and the second enlarged area PA(2). Even in this case, the PIP processor 61 adjusts the positional relationship between the first enlarged area LPA(1) and the second enlarged area PA(2) at their destinations to correspond to the positional relationship between the first subject S(1) and the second subject S(2), as shown in the lower diagram of FIG. 24 . In this way, if the positional relationship between the multiple enlarged areas LPA corresponds to the positional relationship between each subject S corresponding to each enlarged area LPA, it is easy to intuitively determine which subject S corresponds to which enlarged area LPA.

[0127] (Variation 8: Hunting Countermeasure Using Dead Zones) As shown in FIG. 25 , the PIP processing unit 61 of the processor 40 may not execute the overlap reduction process if the movement amount and reduction rate of the enlarged area LPA determined in the overlap reduction process are equal to or less than preset values, even if the conditions are met. The flowchart of FIG. 25 is the same as the flowchart of FIG. 11 except for step S1750. If the PIP processing unit 61 determines in step S1700 that the condition regarding the overlap area OV is met (Y in step S1700), the PIP processing unit 61 proceeds to step S1750. Then, in step S1750, for example, if the movement amount of the enlarged area LPA required to eliminate the overlap area OV exceeds a preset value (Y in step S1750), the overlap reduction process is executed. However, if the movement amount is equal to or less than the preset value (N in step S1750), the overlap reduction process is not executed. By setting a dead zone to limit the movement of the enlarged area LPA when the amount of movement is small, it is possible to suppress hunting, which is frequent and small movements of the enlarged area LPA. Of course, the same applies when the reduction rate is small instead of the amount of movement. The size of the set value is determined to be an appropriate size for the dead zone.

[0128] (Variation 9: Countermeasures against hunting due to detection accuracy) As shown in Figure 26, when repeatedly detecting the enlargement target SP from the image 36, if an index related to the accuracy of detection of the enlargement target SP is equal to or lower than a preset standard, the PIP processing unit 61 of the processor 40 may perform at least one of determining the enlargement target SP and determining the content of the overlap reduction processing based on past detection results. If the detection of the enlargement target SP is unstable, hunting may occur, in which the enlargement area LPA is repeatedly displayed and hidden. The example shown in Figure 26 is an example in which past detection results are used as a countermeasure against hunting due to such detection accuracy.

[0129] As shown in FIG. 26 , the PIP processing unit 61 compares the index of the detection accuracy of the enlargement target SP with a reference value. If the index is equal to or less than the reference value, the PIP processing unit 61 performs PIP display based on past detection results. Specifically, the PIP processing unit 61 performs at least one of determining the enlargement target SP and determining the details of the overlap reduction process, such as determining the insertion position, based on the past detection results. For example, the PIP processing unit 61 records the detection rate as an index of detection accuracy. The detection rate is, for example, the ratio of the number of detections to the number of frames when detecting the enlargement target SP according to the frame rate at which the image sensor 20 acquires the images 36. For example, in the case of detecting an eye SP (E), the past detection result is the history of the detected position of the eye SP (E). If the detection rate of the enlargement target SP is equal to or less than the reference value, the PIP processing unit 61 predicts the position of the enlargement target SP based on the past detection results and detects the predicted area as the enlargement target SP. Furthermore, the PIP processing unit 61 determines the insertion position of the enlargement area LPA so as to avoid the predicted position of the enlargement target SP. This makes it possible to suppress hunting caused by detection accuracy.

[0130] (Variation 10: Determining the part to be enlarged according to the size of the subject) The PIP processing unit 61 of the processor 40 may determine the part of the subject S to be the part to be enlarged SP according to the size of the subject S in the image 36. For example, as shown in FIG. 27 , when the size of the subject S in the image 36 is small, even if the eyes are detected as the part to be enlarged SP, the resolution may be too low, making it difficult to confirm the state of focus. Therefore, as an example, when the size of the subject S is small, the PIP processing unit 61 determines the face, which is larger than the eyes, as the part to be enlarged SP. This makes it easier to confirm the state of focus.

[0131] (Variation 11: Improving Visibility of Enlarged Region) As shown in Fig. 28 , the PIP processing unit 61 of the processor 40 may be capable of adjusting the visibility of the enlarged region LPA separately from the subject region SA. For example, the visibility of the enlarged region LPA is improved by performing brightness correction or color correction on the enlarged region LPA independently of the subject region SA. The example of Fig. 28 shows that the visibility of the enlarged region LPA is improved compared to the regions other than the enlarged region LPA (including the subject region SA) in the image 36.

[0132] (Variation 12: Transparency) As shown in Fig. 29, the overlap reduction process may involve adjusting the transparency of the overlap region OV. For example, if the enlarged region LPA is displayed in front of the subject region SA, the transparency of the enlarged region LPA can be increased to allow the subject region SA to be faintly seen. In this way, the overlap reduction process includes a process of adjusting the transparency.

[0133] In the above embodiment, a numerical index is used as an example of the index representing the degree of overlap ODG, but an index other than a numerical value, such as large, medium, or small, may also be used. In this case, the threshold value TH is also set so that the degree of overlap ODG is equal to or greater than "medium."

[0134] The technology of the present disclosure is not limited to digital cameras, but can also be applied to electronic devices such as smartphones and tablet terminals that have an imaging function.

[0135] The above description allows the understanding of the following techniques. [Supplementary Item 1] A display control device including a processor, wherein the processor acquires an image including a subject, detects from the image at least a part of the subject as an enlargement target in addition to a subject region representing the subject, inserts an enlarged region into a display screen that displays the image, the enlarged region being an enlarged target region representing the enlargement target, and, if an overlapping region where the subject region and the enlarged region overlap occurs on the display screen and a condition is satisfied, performs an overlap reduction process to reduce the overlapping degree of the overlapping region. [Supplementary Item 2] The display control device according to Supplementary Item 1, wherein the processor performs the overlap reduction process by adjusting at least one of the display size and insertion position of the enlarged region. [Supplementary Item 3] The display control device according to Supplementary Item 1 or Supplementary Item 2, wherein, in the overlapping region, the enlarged region is displayed in front of the subject region. [Supplementary Item 4] The display control device according to Supplementary Item 3 ... the condition is specified by a magnitude relationship between a numerical index indicating the overlapping degree and a threshold. [Supplementary Item 5] The display control device according to Supplementary Item 4, wherein the numerical index includes any one of the ratio of the area of ​​the overlapping region to the area of ​​the subject region or the area of ​​the enlargement region, the number of overlapping regions, and the distance between the subject region and the enlargement region. [Supplementary Item 6] The display control device according to Supplementary Item 4 or Supplementary Item 5, wherein the threshold is changeable according to the part of the subject in the overlapping region. [Supplementary Item 7] The display control device according to Supplementary Item 5 or Supplementary Item 6, wherein the numerical index is the ratio of the area of ​​the overlapping region to the area of ​​the subject region or the area of ​​the enlargement region, and the threshold is changeable according to the area of ​​the subject region. [Supplementary Item 8] The display control device according to any one of Supplementary Item 1 to Supplementary Item 7, wherein if the subject region is detected as a rectangular region including the subject, the processor determines at least one of the presence or absence of an overlapping region and whether a condition is satisfied based on the overlap between the rectangular region and the enlargement region. [Supplementary Item 9] The display control device according to Supplementary Item 8, wherein the target region is also detected as a rectangular region including the target to be enlarged.[Supplementary Item 10] The display control device according to any one of Supplementary Items 1 to 9, wherein, when an image includes a plurality of subjects and an enlargement target of at least one of the plurality of subjects is inserted as an enlargement region, the processor determines whether or not there is an overlapping region based on the relationship between the enlargement region and the subject region corresponding to the enlargement region. [Supplementary Item 11] The display control device according to any one of Supplementary Items 1 to 10, when an image includes a first subject and a second subject as subjects and a first enlargement region that is an enlargement region corresponding to an enlargement target of the first subject and a second enlargement region that is an enlargement region corresponding to an enlargement target of the second subject are inserted, the processor determines the positional relationship between the first enlargement region and the second enlargement region based on the positional relationship between the first subject region representing the first subject and the second subject region representing the second subject in the image. [Supplementary Item 12] The display control device according to any one of Supplementary Items 1 to 11, wherein, when a plurality of initial positions are set with priorities assigned as insertion positions of the enlargement region, the processor determines the insertion position in accordance with the priorities if each of the plurality of initial positions satisfies a condition. [Supplementary Item 13] The display control device according to any one of Supplementary Items 1 to 12, wherein, when the subject is a living body, the part detected as the enlargement target is any one of the subject's eyes, face, or head. [Supplementary Item 14] The display control device according to any one of Supplementary Items 1 to 13, wherein a cycle for determining whether to perform the redundancy reduction process corresponds to either a refresh rate of the display screen or a frame rate of the image. [Supplementary Item 15] The display control device according to any one of Supplementary Items 1 to 14, wherein, even if the conditions are satisfied, the processor does not perform the redundancy reduction process if the movement amount and reduction rate of the enlargement region determined in the redundancy reduction process are equal to or less than preset values.[Supplementary Item 16] The display control device according to any one of Supplementary Items 1 to 15, wherein, when repeatedly detecting an object to be enlarged from an image, the processor, if an index relating to the accuracy of detection of the object to be enlarged is equal to or lower than a preset standard, executes at least one of determining the object to be enlarged and determining the content of the overlap reduction process based on past detection results. [Supplementary Item 17] The display control device according to any one of Supplementary Items 1 to 16, wherein the overlap reduction process includes a process of adjusting the transparency of the overlapping region. [Supplementary Item 18] The display control device according to any one of Supplementary Items 1 to 17, wherein the processor determines the part of the subject to be enlarged according to the size of the subject in the image. [Supplementary Item 19] The display control device according to any one of Supplementary Items 1 to 18, wherein the processor is capable of adjusting the visibility of the enlargement region separately from the subject region. [Supplementary Item 20] An imaging device including the display control device according to any one of Supplementary Items 1 to 19, wherein the processor starts displaying the enlarged area when an imaging operation is started. [Supplementary Item 21] The imaging device according to Supplementary Item 20, wherein the imaging operation is a focusing operation. [Supplementary Item 22] The imaging device according to Supplementary Item 20 or Supplementary Item 21, wherein the processor ends displaying the enlarged area when the imaging operation is finished. [Supplementary Item 23] The imaging device according to any one of Supplementary Items 20 to 22, wherein the processor starts or ends displaying the enlarged area based on operation of the release button. [Supplementary Item 24] A method for operating a display control device having a processor, wherein the processor acquires an image including a subject, detects from the image at least a part of the subject as an object to be enlarged in addition to a subject area representing the subject, inserts an enlarged area into a display screen that displays the image, the enlarged area being an enlarged object area representing the object to be enlarged, and, if an overlapping area occurs on the display screen where the subject area and the enlarged area overlap and a condition is satisfied, executes an overlap reduction process that reduces the overlapping degree of the overlapping area.[Supplementary Item 25] An operating program for a display control device having a processor, the operating program causing a processor to perform processes including: acquiring an image including a subject; detecting from the image, in addition to a subject area representing the subject, at least a part of the subject as an object to be enlarged; inserting, on a display screen that displays the image, an enlarged area that is an enlarged object area representing the object to be enlarged; and, when an overlapping area occurs on the display screen where the subject area and the enlarged area overlap and a condition is satisfied, performing an overlap reduction process that reduces the overlapping degree of the overlapping area.

[0136] In the above embodiment, the hardware structure of the control unit, with processor 40 being an example, can use the various processors listed below. The various processors listed above include a CPU, which is a general-purpose processor that functions by executing software (programs), as well as a processor such as an FPGA, whose circuit configuration can be changed after manufacture. FPGAs include dedicated electrical circuits, such as PLDs or ASICs, which are processors with a circuit configuration designed specifically to execute specific processes.

[0137] The control unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple control units may be configured with a single processor.

[0138] There are several possible examples of configuring multiple control units with a single processor. A first example is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple control units. A second example is a form in which a processor is used to realize the functions of an entire system including multiple control units with a single IC chip, as typified by a system on chip (SOC). In this way, the control unit can be configured as a hardware structure using one or more of the various processors described above.

[0139] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0140] The technology of the present disclosure can also be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, the technology is not limited to the above embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure also covers, in addition to programs, storage media that non-temporarily store programs. The storage medium is, for example, a computer-readable non-temporary storage medium such as a USB (Universal Serial Bus) memory, a flexible disk, or a CD-ROM (Compact Disc Read Only Memory). The program may also be provided online via a network such as the Internet. The technology of the present disclosure also covers, in addition to programs, program products. A program product includes any type of product for providing a program. Like a program, a program product may be provided stored on a computer-readable non-temporary storage medium or provided online.

[0141] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0142] The disclosure of Japanese Patent Application No. 2023-173228, filed on October 4, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A display control device having a processor, wherein the processor: acquires an image including a subject; detects from the image a subject area representing the subject, and at least a part of the subject as an object to be enlarged; inserts, on a display screen displaying the image, an enlarged area obtained by enlarging the object area representing the object to be enlarged, into the display screen; and, when an overlap area occurs on the display screen where the subject area and the enlarged area overlap and a condition is satisfied, executes an overlap reduction process to reduce the overlap of the overlap area.

2. The display control device according to claim 1, wherein the processor executes the overlap reduction process by adjusting at least one of the display size and the insertion position of the enlarged area.

3. The display control device according to claim 1, wherein the enlarged area is displayed in front of the subject area in the overlapping area.

4. The display control device according to claim 3, wherein the condition is defined by a magnitude relationship between a numerical index indicating the degree of overlap and a threshold value.

5. The display control device according to claim 4, wherein the numerical indicator includes any one of the following: the ratio of the area of ​​the overlapping area to the area of ​​the subject area or the area of ​​the enlarged area, the number of the overlapping areas, and the distance between the subject area and the enlarged area.

6. The display control device according to claim 4, wherein the threshold value is changeable depending on a part of the subject in the overlapping area.

7. The display control device according to claim 5, wherein the numerical index is a ratio of an area of ​​the overlapping region to an area of ​​the subject region or an area of ​​the enlarged region, and the threshold is changeable according to an area of ​​the subject region.

8. The display control device of claim 1, wherein when the subject area is detected as a rectangular area including the subject, the processor determines at least one of the presence or absence of the overlap area and whether or not the condition is satisfied based on the overlap between the rectangular area and the enlarged area.

9. The display control device according to claim 8, wherein the target area is also detected as a rectangular area including the enlargement target.

10. A display control device as described in claim 1, wherein when the image contains multiple subjects and the enlargement target of at least one of the multiple subjects is inserted as the enlargement area, the processor determines the presence or absence of the overlap area based on the relationship between the enlargement area and the subject area corresponding to the enlargement area.

11. A display control device as described in claim 1, wherein when the image includes a first subject and a second subject as the subjects, and a first enlarged region corresponding to the enlargement target of the first subject and a second enlarged region corresponding to the enlargement target of the second subject are inserted, the processor determines the positional relationship between the first enlarged region and the second enlarged region based on the positional relationship between the first subject region representing the first subject and the second subject region representing the second subject in the image.

12. The display control device according to claim 1, wherein when a plurality of initial positions are set with priority as the insertion position of the enlarged area, the processor determines the insertion position according to the priority if each of the plurality of initial positions satisfies the condition.

13. A display control device according to claim 1, wherein when the subject is a living body, the part detected as the area to be enlarged is any one of the subject's eyes, face, or head.

14. The display control device according to claim 1, wherein a period for determining whether or not to execute the overlap reduction process corresponds to either a refresh rate of the display screen or a frame rate of the image.

15. The display control device according to claim 1, wherein the processor does not execute the redundancy reduction process if the movement amount and reduction ratio of the enlarged area determined in the redundancy reduction process are equal to or less than preset values ​​even if the condition is satisfied.

16. The display control device of claim 1, wherein the processor, when repeatedly detecting the target to be enlarged from the image, if an index relating to the accuracy of the detection of the target to be enlarged is below a predetermined standard, performs at least one of determining the target to be enlarged and determining the content of the redundancy reduction process based on past detection results.

17. The display control device according to claim 1, wherein the overlap reduction process includes a process of adjusting the transparency of the overlapping region.

18. The display control device according to claim 1, wherein the processor determines the part of the subject to be enlarged according to the size of the subject within the image.

19. The display control device according to claim 1, wherein the processor is capable of adjusting visibility of the enlarged region separately from the subject region.

20. An imaging device including a display control device according to any one of claims 1 to 19, wherein the processor starts displaying the enlarged area when an imaging operation is started.

21. The imaging device according to claim 20, wherein the imaging operation is a focusing operation.

22. The imaging device according to claim 20, wherein the processor terminates display of the enlarged area when the imaging operation is terminated.

23. The imaging device according to claim 20, wherein the processor starts or ends the display of the enlarged area based on the operation of a release button.

24. A method for operating a display control device having a processor, wherein the processor: acquires an image including a subject; detects from the image at least a part of the subject as an object to be enlarged in addition to an object area representing the subject; inserts, on a display screen displaying the image, an enlarged area obtained by enlarging the object area representing the object to be enlarged, into the display screen; and, if an overlap area occurs on the display screen where the subject area and the enlarged area overlap and a condition is satisfied, executes an overlap reduction process for reducing the overlap of the overlap area.

25. An operating program for a display control device having a processor, the operating program causing a processor to execute processes including: acquiring an image including a subject; detecting from the image, in addition to a subject area representing the subject, at least a part of the subject as an object to be enlarged; inserting, on a display screen displaying the image, an enlarged area obtained by enlarging the object area representing the object to be enlarged, into the display screen; and, when an overlap area occurs on the display screen where the subject area and the enlarged area overlap and a condition is satisfied, executing an overlap reduction process to reduce the overlap of the overlap area.

Images