Display method

Abstract

The present invention provides a method for displaying an image that is a part of a cut-out photographic image and that is more useful for switching the cut-out image. A method for displaying an image captured by a photographic device comprises: a setting step of automatically setting a plurality of setting areas or candidates for setting areas in a reference area of ​​a photographic area that is a reference image; a selection step of selecting a recording area as an area of ​​a recorded image to be recorded from the plurality of setting areas; and a switching step of reselecting the recording area from the plurality of setting areas and switching the recording area after the selection step is performed.

Description

Technical Field

[0001] One embodiment of the present invention relates to a method for displaying an image. Background Art

[0002] A technology has been developed for switching and storing an image of an arbitrary subject among a plurality of subjects in an image captured by a single imaging device. As an example of this technology, for example, the technology described in Patent Document 1 can be cited.

[0003] In the technology described in Patent Document 1, a plurality of different image areas are set in the photographing range of the photographing device, and an image of one of the areas is recorded. In addition, in the technology described in Patent Document 1, the area of ​​the image to be recorded can be changed (switched) among the plurality of areas. Thus, for example, it is possible to obtain an image in which each of the plurality of subjects in the photographed image is zoomed in while the photographing device is fixed.

[0004] Previous technical literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2016-92467 Summary of the invention

[0007] Technical issues to be solved by the invention

[0008] An embodiment of the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a method of cutting out a part of a photographed image and switching an image display method of the cutout image, which is more useful.

[0009] Means for solving technical problems

[0010] In order to achieve the above-mentioned purpose, one embodiment of the present invention is a display method, which displays an image captured by a photographic device, and the display method comprises: a setting process, in which a plurality of setting areas or candidates for setting areas are automatically set in a reference area of ​​a photographic area serving as a reference image; a selection process, in which a recording area as an area of ​​a recorded image to be recorded is selected from a plurality of setting areas; and a switching process, in which, after implementing the selection process, a recording area is reselected from a plurality of setting areas and the recording area is switched.

[0011] Furthermore, the setting step may further include: a first setting step of automatically setting the candidate in the reference area; and a second setting step of setting the setting area at the position where the candidate is set according to the user's operation on the candidate.

[0012] Furthermore, in the setting process, a characteristic part that visually changes in the reference image may be detected, and the setting area or the candidate may be automatically set at the position of the detected characteristic part in the reference area.

[0013] Furthermore, in the setting step, the photographic scene of the reference image may be recognized based on the reference image, and the setting area or the candidate may be automatically set at a position corresponding to the recognized photographic scene in the reference area.

[0014] Furthermore, in the setting step, the setting area may be automatically set at a position in the reference image where a feature for determining the imaging condition of the imaging device satisfies a reference.

[0015] Here, the photographing condition may be at least one of a focus position, an exposure amount, and a white balance in the reference image.

[0016] Furthermore, in the setting step, the setting area or the candidates may be automatically set based on the position information of the setting area stored in advance.

[0017] Here, the position information may include at least one of position information of a setting area set in a setting process performed in the past and position information of a setting area determined based on a user's operation.

[0018] Furthermore, the method may further include an adjusting step of adjusting at least one of the position and the size of the setting area when the size or the position of the reference area changes.

[0019] Furthermore, one embodiment of the present invention is a display method for displaying an image captured by a photographic device, the display method comprising: a setting process of setting a plurality of setting areas in a reference area of ​​a photographic area serving as a reference image; a selection process of selecting a recording area as an area of ​​a recorded image to be recorded from the plurality of setting areas; a switching process of reselecting the recording area from the plurality of setting areas and switching the recording area after implementing the selection process; a recording process of recording a dynamic image of the recorded image; a display process of displaying a set image as an image of the set area; and a change process of performing a change process related to a change in the number of displayed set images during the implementation of the recording process.

[0020] Furthermore, the change process may include a process of adding a set area and increasing the number of displayed areas in accordance with an increase in the number of subjects in the reference image.

[0021] Furthermore, the change process may include a process of stopping the display of the setting images of the setting areas satisfying the stop condition and reducing the number of displays.

[0022] Furthermore, the pause area, which is the setting area where display of the setting image is paused, may be stored during the recording process. In this case, the change process may include a process of redisplaying the setting image of the stored pause area based on a redisplay instruction from the user.

[0023] Furthermore, the change process may include a process of outputting proposal information related to the change of the display quantity.

[0024] Furthermore, when the number of subjects in the reference image exceeds the upper limit of the number of set areas, a change process related to a decrease in the number of displayed areas may be executed in the change step. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a perspective view showing an example of the appearance of the imaging device involved in one embodiment of the present invention.

[0026] Figure 2 It is a rear view showing the rear side of the imaging device involved in one embodiment of the present invention.

[0027] Figure 3 This is a block diagram showing the structure of a photographing device according to one embodiment of the present invention.

[0028] Figure 4 This is a diagram to explain electronic camera shake correction.

[0029] Figure 5 It is a diagram showing the relationship between the reference area and the set area.

[0030] Figure 6 This is a diagram showing the positional relationship between the reference image and the recorded image.

[0031] Figure 7 The diagram shows recorded images before and after switching in a moving image showing recorded images.

[0032] Figure 8 This is a diagram showing a reference image when a new subject enters the photographic area.

[0033] Fig. 9 This is a diagram showing a state in which a moving area follows a moving subject.

[0034] Fig.10 This is a diagram showing the flow of the image display process.

[0035] Fig.11 This is a diagram showing the transition of the UI screen during the setting process.

[0036] Fig.12 This is a diagram showing a state where the setting area is set when the last used pattern is selected.

[0037] Fig.13 This is a diagram showing a selection screen for a preset pattern.

[0038] Fig.14 This is a diagram for explaining user operations related to the position of the setting area.

[0039] Fig.15 It is a diagram showing a setting area set based on the AF position determination pattern.

[0040] Fig.16 It is a diagram showing characteristic parts in the reference image.

[0041] Fig.17 This is a diagram showing a state where a setting area is set at a characteristic portion.

[0042] Fig.18 A diagram showing an example of a display screen when the display step is performed.

[0043] Fig.19 This is a diagram showing an example of a display screen when the second display step is performed.

[0044] Fig. 20 It is a diagram showing a modified example of the display screen when the second display step is implemented.

[0045] Fig.21 This is a diagram showing an example of a screen when the number of standby areas reaches the maximum display number of standby images (part 1).

[0046] Fig. 22 This is a diagram showing the flow of the display process when the number of standby areas reaches the maximum display number of standby images.

[0047] Fig.23 This is a diagram showing an example of a screen when the number of standby areas reaches the maximum display number of standby images (part 2).

[0048] Fig.24 This is a diagram showing a modified example of the screen when the number of standby areas reaches the maximum display number of standby images.

[0049] Fig.25 This is a diagram showing the flow of processing when performing a recording step.

[0050] Fig.26 This is a diagram showing an example of a screen before a change process is implemented.

[0051] Fig. 27 This is a diagram showing an example of a screen when the number of displayed images is increased due to a change in the process.

[0052] Fig.28This is a diagram showing an example of a screen when a moving area overlaps with another set area.

[0053] Fig.29 This is a diagram showing an example of a screen when overlapping movement areas and other setting areas are merged into one setting area.

[0054] Fig.30 This is a diagram showing an example of a screen when display of a setting image of a certain setting area is stopped.

[0055] Fig.31 This is a diagram showing an example of a screen displaying proposal information for urging a change in a reference area. DETAILED DESCRIPTION

[0056] A preferred embodiment of the present invention (hereinafter referred to as the present embodiment) will be described in detail with reference to the drawings.

[0057] This embodiment involves using Figure 1 to Figure 3 The present invention is a method for displaying an image of the photographic device 10 shown in the figure. However, the embodiment described below is only an example given to facilitate understanding of the present invention and does not limit the present invention. That is, the present invention can be changed or improved from the embodiment described below as long as it does not deviate from its purpose. In addition, the present invention includes its equivalents.

[0058] [Basic structure of the photographic device]

[0059] The imaging device 10 is, for example, a digital camera for capturing images.

[0060] Here, unless otherwise specified, "image" refers to a real-time image (live view image), that is, an image captured in real time. In addition, the image in this embodiment is mainly a dynamic image. That is, the photography device 10 captures dynamic images at a predetermined frame rate.

[0061] The photographing device 10 is, for example, a lens-interchangeable type. Figure 1 and Figure 2 As shown, the imaging device 10 includes an imaging device body 12 and an imaging lens 14. The imaging lens 14 is replaceably mounted on a mount 13 of the imaging device body 12. However, the present invention is not limited thereto, and the imaging device 10 may also be a lens-integrated type.

[0062] (Photographic lens)

[0063] like Figure 3 As shown, the photographic lens 14 includes an optical unit 17 , an aperture 20 , a zoom drive section 21 , a focus drive section 22 , and an aperture drive section 23 .

[0064] The optical unit 17 includes an optical device 18 for zooming (eg, a zoom lens). By moving the optical device 18 in the direction of the optical axis L1, the imaging area of ​​the imaging device 10 is enlarged or reduced to change the zoom. That is, the imaging device 10 has an optical zoom function based on the optical device 18.

[0065] Furthermore, the photographing device 10 of the present embodiment has an electronic zoom (digital zoom) function for electronically processing and magnifying a subject in an image, in addition to the optical zoom function.

[0066] The optical unit 17 also includes an optical device 19 for focusing (for example, a focus lens). By moving the optical device 19 in the direction of the optical axis L1, the focus position of the photographic lens 14 changes, thereby adjusting the focus.

[0067] In addition, the photographing device 10 of this embodiment has an auto focus (AF) function. That is, when the user performs a predetermined operation (e.g., an instruction operation to start recording an image) during image photography, the focus drive unit 22 is operated to move the optical device 19. As a result, the focus position is automatically adjusted so that a predetermined portion of the image is focused.

[0068] Furthermore, the optical unit 17 includes a wide-angle lens, an ultra-wide-angle lens, a 360-degree lens, or an anamorphic lens. As a result, the photographing device 10 can capture images with a wide field of view in the lateral direction (width direction) of the photographing device 10. The field of view of the photographing device 10 changes with the execution of optical zoom, etc. Furthermore, the maximum field of view (total field of view) of the photographing device 10 is determined by the design specifications of the optical unit 17 and the imaging element 40 described later, etc.

[0069] That is, the optical unit 17 provided in the imaging device 10 is not limited to one, and a plurality of optical units 17 having different viewing angles may be provided.

[0070] The diaphragm 20 has a variable opening shape, and the opening shape is adjusted by the diaphragm driving unit 23. By adjusting the opening shape, the aperture amount with respect to the incident light to the photographic lens 14 changes, thereby adjusting the exposure amount.

[0071] (Photographic device body)

[0072] like Figure 1 and Figure 2 As shown, the photographing device body 12 includes an operation unit including an operation button 30. In addition, a release button 26 as one of the operation units is arranged on the upper surface of the photographing device body 12. When the release button 26 is pressed, the image captured by the photographing device 10 or the image cut out therefrom (strictly speaking, the recorded image described later) starts to be recorded.

[0073] Here, the photographing area of ​​the photographing device 10 is determined at the time of receiving the recording start instruction from the user, that is, the range of the field of view of the photographing device 10. In addition, the photographing area changes according to, for example, the on / off state of the electronic hand shake correction, etc. When the electronic hand shake correction is in the off state at the time of receiving the recording start instruction, the photographing area is usually a full-size (total field of view) photographing area.

[0074] The user's operation of instructing the start of video recording is not limited to pressing release button 26 , and may be an operation of touching a predetermined position on rear display 28 described later.

[0075] Hereinafter, the image captured by the imaging device 10 at the field of view angle at that time is referred to as a “reference image.” Furthermore, the imaging area of ​​the reference image is referred to as a “reference area.”

[0076] A Figure 2 The display shown (hereinafter also referred to as the back display 28 ) is composed of, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electroluminescence) display, an LED (Light Emitting Diode) display, or electronic paper.

[0077] The display 28 on the rear side displays a reference image and a recorded image described later, etc. As described above, the photographing device 10 having a display has a function of displaying an image, and is therefore used as an image display device.

[0078] Furthermore, the rear display 28 can display information other than images, such as suggestions or advice related to photography (see Fig.18 ).

[0079] In addition, if Figure 2 and Figure 3 As shown, the photographing device body 12 of this embodiment includes an electronic viewfinder ( Figure 3 In the figure, it is described as EVF) 29. The same image can be displayed on the rear display 28 and the electronic viewfinder 29, or different images can be displayed. That is, the photographic device does not need to include the electronic viewfinder 29. Furthermore, the image can be displayed on the electronic viewfinder 29, but not on the rear display 28.

[0080] Furthermore, a touch panel 36 for detecting user operations is provided inside the rear display 28 or on the exposed surface of the display 28. For example, when the user touches a predetermined position of the rear display 28, the touch panel 36 detects the touched position and outputs the detection result to the control unit 46 described later.

[0081] like Figure 3 As shown, the photographic device body 12 is also provided with a shutter 38 , an imaging element 40 , an analog signal processing circuit 44 , a control unit 46 , an internal memory 50 , a card slot 52 and a buffer memory 56 .

[0082] The imaging element 40 is an image sensor that receives light passing through the photographic lens 14, converts the received light image into an electrical signal (image signal), and outputs the image. As the imaging element 40, a solid imaging element such as a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor Image Sensor) can be used.

[0083] The control unit 46 controls each unit of the imaging device 10 and executes various processes related to capturing, displaying, and recording images. Figure 3 As shown, the control unit 46 includes a controller 47 and an image processing unit 48 .

[0084] The control unit 46 has one or more processors, and the control unit 46 is formed by the cooperation of the processor and the control program. The processor can be composed of, for example, a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), or other ICs (Integrated Circuits). Alternatively, these can be combined to form a processor.

[0085] Furthermore, as represented by SoC (System on Chip) or the like, the processor may be configured by a single IC (Integrated Circuit) chip to constitute the entire function of the control unit 46 including the controller 47 and the image processing unit 48 .

[0086] The hardware structure of each processor listed above can be implemented by a circuit (Circuitry) formed by combining circuit elements such as semiconductor elements.

[0087] The controller 47 controls each unit of the imaging device 10 based on user operations or according to a predetermined control program. For example, the controller 47 controls the imaging element 40 and the analog signal processing circuit 44 so as to capture an image at a predetermined frame rate.

[0088] The controller 47 controls the driving units 21 to 23, the imaging element 40, the analog signal processing circuit 44, etc., so that the photographing conditions become the desired conditions. The photographing conditions include focusing (focus), exposure, white balance, etc. That is, the automatic focusing (AF) function, the automatic exposure (AE) function, and the automatic white balance (AWB) function, which are the automatic adjustment functions of the photographing device 10, are realized by the controller 47.

[0089] Furthermore, upon receiving a recording start instruction from the user, the controller 47 controls the image processing unit 48 to start recording the image.

[0090] The image processing unit 48 converts the analog image signal sent from the analog signal processing circuit 44 into digital image data, and performs gamma correction, white balance correction, etc. on the digital image data. The processed digital image data is compressed in a compression format in accordance with a predetermined standard.

[0091] During the image capturing, the image processing unit 48 generates compressed digital image data at a predetermined frame rate and acquires an image (strictly speaking, a frame image) from the data. The image acquired at this time corresponds to the reference image captured in the reference area.

[0092] Furthermore, the image processing unit 48 cuts the recorded image from the reference image under the control of the controller 47. The recorded image is an image having a smaller viewing angle than the reference image, and is an image of a part of the reference area. The recorded image is generated at the same frame rate as the reference image, and is displayed on, for example, the rear display 28 or the electronic viewfinder 29.

[0093] Then, the recorded image is recorded on the recording medium. As a result, an image file of the recorded image is created. At this time, the resolution (number of pixels) and aspect ratio of the recorded image are determined according to the recording format (movie format) of the recorded image.

[0094] In addition, the recording format is not limited to this and can be freely determined.

[0095] Furthermore, while the electronic image stabilization is effective (ON), Figure 4As shown, the image processing unit 48 extracts an extracted image slightly smaller in size than the image before correction from the image before correction captured at the total field of view. Then, if the extracted image is offset due to vibration or the like, the image processing unit 48 performs electronic hand shake correction to offset the position of the extracted area in the image before correction (for example, Figure 4 , shifted from the position of the solid line to the position of the dotted line).

[0096] In addition, while the electronic image stabilization is on, the extracted image becomes the reference image.

[0097] In the following, unless otherwise specified, the operations and processes of the controller 47 and the image processing unit 48 are described as the operations and processes of the control unit 46 .

[0098] The memory card 54 which can be inserted and removed into the internal memory 50 and the card slot 52 built in the photographing device body 12 is a recording medium, for example, recording images.

[0099] In addition, the internal memory 50 and the memory card 54 may be provided in a device (external) different from the imaging device body 12. In this case, the control unit 46 may record the recorded image in an external recording medium by wire or wirelessly.

[0100] The buffer memory 56 functions as a working memory of the control unit 46 .

[0101] [Regarding recording and switching of recorded images]

[0102] The photographing device 10 records the recorded image cut out from the reference image on the recording medium. In addition, the photographing device 10 can switch the recorded image (switching) by changing the cutout area in the reference image. Figure 5 Provide explanation.

[0103] In addition, Figure 5 In the figure, for the sake of convenience, the illustration of the subject in the image is omitted.

[0104] When the photographing device 10 starts shooting, the reference image is displayed on the display (for example, the display 28 on the back). At this time, usually, the image of the total field angle becomes the reference image. Figure 5 The area around the outermost edge of the image is the reference area As. Also, during the electronic hand shake correction on state, the extracted image extracted from the image of the total field angle becomes the reference image. Figure 5 The area surrounded by the dotted box in becomes the reference area At.

[0105] If the user performs a predetermined operation before or during the start of shooting, the control unit 46 sets the setting area Ar in the reference areas As and At. The setting area Ar is a part of the reference areas As and At. For example, the subject in the reference image is reflected in the setting area Ar (refer to Figure 6 ).

[0106] Furthermore, the position and size of the setting area Ar can be adjusted automatically or based on user operations. The user can change the position and size of the setting area Ar where the subject is reflected, for example, according to the position and size of the subject in the reference image, by performing a predetermined operation on the screen.

[0107] Here, in this embodiment, the aspect ratio of the setting area Ar is determined to be a value corresponding to the recording format (animation format) of the recorded image. Therefore, when the size of the setting area Ar is changed, the size is changed while the aspect ratio of the setting area Ar is constant. However, the aspect ratio of the setting area Ar may be determined independently of the recording format of the recorded image. In this case, the size of the setting area Ar may be changed while the aspect ratio is changed.

[0108] like Figure 5 As shown in FIG. 1 , when the setting area Ar is set, at that point in time, a mark f is displayed in the reference image displayed on the display.

[0109] The mark f indicates the position of the setting area Ar in the reference image, and includes the boundary of the setting area Ar. The boundary of the setting area Ar is set in order to identify (distinguish) the setting area Ar from its surroundings.

[0110] As the mark f including the boundary, for example, there can be cited graphic objects such as a frame surrounding the setting area Ar and a pointer indicating a representative position of the setting area Ar. Furthermore, the difference in brightness, chromaticity, and shade between the setting area Ar and its surroundings also corresponds to the mark f. In addition, as long as the setting area Ar and its surroundings can be distinguished, it can also be included in the category of the mark f.

[0111] The following description will take the mark f composed of a colored rectangular frame as an example.

[0112] like Figure 5 As shown in FIG. 1 , a plurality of setting areas Ar can be set in the reference image. In this case, a mark f is displayed in each setting area Ar. In addition, the identification information ( Figure 5 In the figure, the number displayed on the upper right of the mark f) is displayed on each mark f.

[0113] The identification information is information that the control unit 46 assigns individually to each mark f according to a predetermined assignment rule. The identification information is displayed together with the mark f, so that the user can distinguish between the marks f.

[0114] The identification information is not limited thereto, and may be a number, a letter, or a symbol, or may be a display color of the mark f distinguished for each mark f. When a number is used as the identification information, a serial number is preferably used.

[0115] When the setting of the set area Ar is completed, the user selects a recording area Ap as an area for recording images from the set multiple set areas Ar. The control unit 46 selects the recording area Ap from the multiple set areas Ar according to the user's selection operation. Thus, the set area Ar of the image in the recording area, that is, the recording area Ap, is determined.

[0116] When the recording area Ap is determined, the control unit 46 displays the mark f indicating the position of the recording area Ap and the mark f indicating the positions of other setting areas Ar in different forms, among the marks f for each setting area Ar displayed in the reference image. For example, the display form of the mark f may include the color, brightness, density (degree of shading), shape, thickness and type of edge line, and presence or absence of blinking display of the mark f.

[0117] exist Figure 5 In the illustrated case, the display color and thickness of the mark f indicating the position of the recording area Ap are different from those of the other marks f.

[0118] When the user issues a recording start instruction after determining the recording area Ap, recording of the image of the recording area Ap (hereinafter referred to as a recorded image) starts. At this point in time, the reference area (ie, the orientation and viewing angle of the imaging device 10) is determined.

[0119] In addition, determining the reference area means that the reference area is determined once at the time when the recording start instruction is received. Moreover, after the reference area is determined, the user can also freely change the reference area. That is, when the user changes the reference area, as described later, it is preferred to change the position of the set area according to the position of the subject.

[0120] As described above, by recording a part of the reference image as a recorded image, for example, a zoomed image of a portion of interest in the reference image can be recorded.

[0121] Although the recorded image has a smaller field angle than the reference image, it can be a high-quality image as long as the reference image is of high quality (for example, as long as the number of pixels is 10 million or more).

[0122] There is no particular limitation on the number of pixels of the reference image, but the lower limit is preferably 10 million or more, and more preferably 60 million or more. Furthermore, the upper limit of the number of pixels is preferably 1 billion or less, and more preferably 500 million or less. If the number of pixels is greater than the lower limit, the visual recognition of the recorded image is guaranteed. Furthermore, if the number of pixels is less than the upper limit, the amount of data of the reference image is reduced, and the processing speed of the control unit 46 can be increased.

[0123] Furthermore, in the present embodiment, after selecting one setting area Ar from among the plurality of setting areas Ar to be determined as the recording area Ap, the recording area Ap can be switched to another setting area Ar during image recording. As a result, the image of the recording area Ap before switching and the image of the recording area Ap after switching can be recorded separately. Figure 6 The details are explained.

[0124] The photographing device 10 photographs a plurality of subjects (people) in a certain scene (position), and Figure 6 As shown in FIG. 1 , a setting area Ar is set for each subject. In this case, one setting area Ar selected by the user from among the plurality of setting areas Ar becomes the recording area Ap. Figure 6 In the case shown, four setting areas Ar are set, and the setting area Ar surrounded by the mark f of the identification information "1" is the recording area Ap.

[0125] When the user issues a recording start instruction in the above-described situation, an image of the set area Ar as the recording area Ap is recorded at that point in time.

[0126] Then, if the user instructs to change the recording area Ap to another set area Ar (for example, set area Ar surrounded by a mark f of identification information "2") during image recording, the recording area Ap is reselected and the recorded image is switched.

[0127] Thus, in the present embodiment, during the image recording, the recorded image can be switched by reselecting the recording area Ap. Thus, it is possible to capture a zoomed image of each of the multiple objects without providing a separate camera for each of the multiple objects.

[0128] Furthermore, when the recorded image is switched, the control unit 46 records the recorded image before the switching and the recorded image after the switching, and combines these images to create a dynamic image. Figure 7 As shown, it is possible to acquire a file (image file) of a dynamic image in which a subject in an image changes as the recorded image is switched.

[0129] However, the present invention is not limited thereto, and the recorded image before switching and the recorded image after switching may be recorded as separate document data without being combined. Furthermore, the reference image and the waiting area described later may also be recorded as separate document data.

[0130] In this embodiment, if Figure 5 and 6 As shown, for each of the multiple setting areas Ar including the recording area Ap, a mark f indicating the position in the reference image is displayed. As a result, the user can grasp the position of each setting area Ar during shooting. That is, from the perspective of easily grasping the position of the setting area Ar, the display method of this embodiment improves convenience (ease of use). In a structure in which the recording area Ap can be switched among multiple setting areas Ar, such an effect is particularly effective. That is, by displaying the mark f for each setting area Ar, it is easy to grasp which image in the reference image the recorded image after switching is.

[0131] To supplement the setting area Ar, for example, when the number of subjects in the reference image changes, the number of setting areas Ar (set number) can be changed automatically or based on user operation according to the change. Figure 8 As shown, when a new subject enters the reference image, a new set area Ar can be added at the position of the subject. Conversely, when the subject in the reference image moves outside the reference image, the set area Ar as the image area of ​​the subject can be automatically deleted.

[0132] Furthermore, when a certain set area Ar (for example, Figure 8 In the example, when the subject in the set area Ar surrounded by the mark f of the identification information "5" moves, the set area Ar can be displaced to follow the movement of the subject. In this way, the subject moving in the reference image can be tracked. Here, the tracking of the subject can be achieved by a known subject detection technology based on image analysis.

[0133] Furthermore, when the set area Ar for tracking the moving subject is the recording area Ap, the image of the moving subject can be recorded.

[0134] Hereinafter, the set area Ar for tracking the moving subject is referred to as a moving area Am. The moving area Am is not limited to an area that moves following the moving subject, and may be an area that moves in a predetermined direction at a constant speed or an irregular speed.

[0135] [Image display process]

[0136] refer to Fig.10, the processing flow (hereinafter referred to as the image display flow) executed by the control unit 46 of the photographic device 10 is described.

[0137] When the user turns on the power of the photographing device 10, the image display process starts. When the image display process starts, the user performs initial adjustment of the photographing device 10. In the initial adjustment, for example, the resolution (number of pixels) and aspect ratio of the recorded image are adjusted by specifying the recording method. This adjustment is performed, for example, when the photographing device 10 is first started, and the adjustment content is retained thereafter.

[0138] The above adjustment can be performed by the device manufacturer before shipping the imaging device 10. Furthermore, in order to modify the resolution and aspect ratio that have been adjusted once, the user can readjust these values ​​after starting the imaging device 10.

[0139] In the image display process, first, the control unit 46 starts a photographing process (S001). The photographing process is a process of photographing a reference image in the reference area at that time. With the start of photographing, the reference image in real time is preferably displayed as a live view image on the rear display 28 or the like. The photographing process continues until the user instructs to end the photographing or turns off the power of the photographing device 10.

[0140] After that, the control unit 46 implements a setting process (S002). The setting process is a process of automatically setting a plurality of setting areas Ar in the photographic area of ​​the reference image. In the setting process, the initial position and initial size of each setting area Ar are set, and then the position and size of each setting area Ar can be automatically changed based on user operation or according to a prescribed rule.

[0141] After the setting process is completed, the control unit 46 implements a selection process (S003). The selection process is a process of selecting a recording area Ap from a plurality of setting areas Ar set in the setting process based on a selection operation by the user.

[0142] When the selection process is performed, the control unit 46 may display images of each of the plurality of setting areas Ar (hereinafter referred to as “setting images”) on the display. In this case, the user can confirm the setting images for each of the plurality of setting areas Ar.

[0143] After that, when the user issues a recording start instruction (S004), the control unit 46 starts a recording process (S005). The recording process is a process of recording a recording image as an image of the selected recording area Ap on a recording medium. The recording image is recorded in a predetermined recording format (moving image format).

[0144] Then, as described above, the reference area is determined at the time of receiving the recording start instruction which is the trigger for starting the recording process.

[0145] Furthermore, during the recording process, the control unit 46 performs a display process (S006). The display process is a process of displaying a plurality of setting images including the recording image, the reference image at that time point, and the mark f in the reference image on the display. The user can observe these images to confirm the position of each setting area Ar in the reference image and understand which setting area Ar is the recording area Ap.

[0146] Furthermore, during the recording process, the control unit 46 performs a control process (S007). The control process is a process for executing a control process related to the change of the reference area determined at the time of receiving the recording start instruction. In the present embodiment, the control process includes a process for controlling the change of the reference area, for example, a control process related to zooming (optical zoom) by the optical device 19.

[0147] Furthermore, during the recording process, if the user instructs to reselect the recording area Ap (S008), the control unit 46 implements a switching process (S009). The switching process is a process of reselecting the recording area Ap from a plurality of set areas Ar and switching the recording area Ap according to the reselection instruction of the user after the selection process is implemented. When the switching process is implemented, the image of the recording area after the switching (recorded image) is recorded in the subsequent recording process.

[0148] Then, until the user instructs to end the video recording, a series of steps from the recording step (ie, S005 to S009) are repeatedly performed (S010). On the other hand, if there is an instruction to end the video recording, the recording step is ended at that point.

[0149] Thereafter, if there is an instruction to end shooting or the power of the photographing device 10 is turned off ( S011 ), the image display process is terminated at that point.

[0150] [About setting process]

[0151] The setting process in the image display flow is described.

[0152] In this embodiment, when the setting process is executed, the display 28 on the back side displays Fig.11 The first UI (User Interface) screen 101 is shown. The user selects either "recorded pattern" or "automatic setting pattern" as the setting pattern of the setting area Ar, and presses the button indicating the selected pattern among the buttons Bt1 and Bt2 on the UI screen 101.

[0153] In the setting process when selecting the recording pattern, the control unit 46 reads the position information of the setting area Ar pre-stored in the imaging device 10. Then, the setting area Ar is automatically set at the position indicated by the read position information. At this time, in order to make it easier for the user to observe the mark f, it is more preferable to highlight the image within the mark f in the reference image. For example, in the reference image, the brightness of the image other than the image within the mark f (that is, within the boundary of the setting area Ar) can be reduced, or grayed out (refer to Fig.18 ).

[0154] In this embodiment, if the recording pattern is selected, Fig.11 As shown, the second UI screen 102 is displayed on the rear display 28. The user selects any one of the plurality of pre-prepared recorded patterns and presses the button representing the selected pattern among the buttons Bt3, Bt4, and Bt5 on the second UI screen 102. For example, "Last used pattern", "Preset pattern", and "User input pattern" can be selected as the recorded pattern.

[0155] On the other hand, in the setting process when the automatic setting pattern is selected, the control unit 46 automatically sets the setting area Ar at a predetermined position in the reference image based on the reference image during the imaging.

[0156] In this embodiment, if the automatic pattern setting is selected, Fig.11 As shown, the third UI screen 103 is displayed. The user selects any one of the multiple automatic setting patterns prepared in advance, and presses the button representing the selected pattern among the buttons Bt6, Bt7, and Bt8 on the third UI screen 103. As the automatic setting pattern, for example, a "scene recognition pattern", a "subject detection pattern", and an "AF position determination pattern" can be selected. In addition, when the pattern is automatically set, it is preferred that the respective setting areas Ar do not overlap with each other at the start point of the setting process, and the mark f is displayed in a state where they do not overlap with each other. Thereby, it becomes easy to identify the multiple marks f, and the setting of the setting area Ar in the setting process becomes easy.

[0157] Hereinafter, a plurality of recording patterns and a plurality of automatic setting patterns will be described respectively.

[0158] (Last used pattern)

[0159] When the last used pattern is selected, the control unit 46 reads the position information of the setting area Ar set in the setting process in the last (immediately before) image display process. The position information indicates the positions of the plurality of setting areas Ar set in the setting process in the last image display process, and is stored in, for example, the internal memory 50 of the photographing device 10. The stored position information is preferably retained even after the power of the photographing device 10 is turned off.

[0160] In addition, the size information of the setting area Ar may be stored together with the position information of the setting area Ar. In this case, when the last used pattern is selected, the position information and the size information may be read together.

[0161] Then, if Fig.12 As shown, the control unit 46 automatically sets the setting area Ar at the position indicated by the read position information, that is, the same position as the setting position in the previous image display process. When the image is shot with the same composition as the previous image display process (for example, when the image is shot at the same position and the same angle), this last used pattern is effective.

[0162] In addition, in the above-mentioned embodiment, the position information read indicates the position of the setting area Ar set in the last (immediately before) image display process, but it is not limited to this. For example, the position information of the setting area Ar set in several setting processes implemented in the past can also be stored separately. In this case, the setting area Ar of this time can be set at the position of the setting area Ar set in the setting process specified by the user in the stored position information.

[0163] (Preset pattern)

[0164] When the preset pattern is selected, the control unit 46 reads the position information of the setting area Ar arranged in a predetermined fixed arrangement pattern. The fixed arrangement pattern is registered in advance in the imaging device 10 or newly registered when the program is updated after purchase.

[0165] If the position information related to the fixed configuration pattern (pre-set pattern) is read, then Fig.13 The fourth UI screen 104 shown is displayed on the rear display 28. In the fourth UI screen 104, multiple types of patterns ( Fig.13 The user selects any one of the preset patterns A to D in the image processing unit. The control unit 46 automatically sets the setting area Ar at a position corresponding to the preset pattern selected by the user. When an image is captured with a determined composition (for example, when an image is captured at a determined position), if there is a pattern corresponding to the composition, the preset pattern is effective.

[0166] The number of types of preset patterns and the number and arrangement positions of the setting areas Ar in each preset pattern are not limited thereto and can be determined arbitrarily.

[0167] (User input pattern)

[0168] When the user input pattern is selected, the control unit 46 reads the position information of the setting area Ar determined based on the user's operation. The user's operation here is an input operation for registering the position of the setting area Ar, which is performed in the previous stage of the setting process (for example, the stage of initial registration related to the pattern). The above-mentioned input operation is performed, for example, by Fig.14 The user touches and drags the inner area of ​​the mark fx for area setting displayed on the input screen 105. Thus, the display position of the mark fx after dragging is registered as the position of the setting area Ar.

[0169] The control unit 46 automatically sets the setting area Ar at the position indicated by the read position information, that is, at the setting position determined based on the user's input operation. Since the above-mentioned pre-set pattern cannot be used, when a pattern is prepared based on the user's input, this user input pattern is effective.

[0170] In addition, when registering the position of the setting area Ar, the size of the setting area Ar may also be registered. That is, information indicating the position and size of the setting area Ar determined based on the user's input operation may also be stored. In this case, the plurality of setting areas Ar are set at the positions and sizes registered as the user input patterns, respectively.

[0171] (Scene recognition pattern)

[0172] In the setting process when the scene recognition pattern is selected, the control unit 46 recognizes the photographic scene of the reference image based on the reference image. The photographic scene of the reference image is, for example, the type of scenery, event, subject, scene (for example, daytime scenery or night scene), and condition (for example, weather, etc.) captured as the reference image.

[0173] In addition, as a method for identifying a photographic scene based on a reference image, a known scene recognition technology can be used. For example, a method for determining whether the brightness or hue of an image in a specified area is within a preset range can be cited. In addition, machine learning can be implemented in which the correct label representing the photographic scene of the image and the data of the image are used as a learning data set, and the photographic scene of the reference image can be recognized by a scene recognition model obtained as a result of the learning.

[0174] (Subject detection pattern)

[0175] In the setting process when the subject detection pattern is selected, the control unit 46 detects the subject in the reference image and automatically sets the setting area Ar at the position of the detected subject. The subject in the reference image is a subject other than the scenery in the reference image and is a subject that can be detected by the control unit 46, such as a person, an animal, and a vehicle.

[0176] When detecting a subject, a well-known template matching technique can be applied. For example, the image of the subject can be pre-stored as a template image, and the template image can be compared with each part in the reference image, and the image similar to the template image can be detected as the subject. Alternatively, a model that recognizes the subject in the image through machine learning can be constructed, and the model can be used to detect the subject in the reference image. As an algorithm for the subject recognition model, well-known algorithms such as YOLO (You Only Look Once) or R-CNN (Region with Convolution Neural Network) can be used.

[0177] In the subject detection pattern, if Figure 6 As shown, the control unit 46 can set the set area Ar for each detected subject. At this time, each set area Ar can contain the entire corresponding subject, or a part of the corresponding subject (for example, the upper body including the face).

[0178] According to the above subject detection pattern, the setting area Ar can be automatically set for the subject in the reference image without the user having to worry. In this embodiment, the subject detection pattern is effective when recording the image of the subject in the reference image in order to record the image of the recording area Ap selected from multiple setting areas Ar.

[0179] (AF position determination pattern)

[0180] In the setting process when selecting the AF position determination pattern, the control unit 46 determines the position in the reference image where the feature for determining the focus of the photographic device 10 satisfies the reference, and automatically sets the setting area Ar at the determined position. The feature for determining the focus is, for example, a feature for determining the focus position within the reference image. To give a specific example, when the autofocus is a phase difference AF (Auto Focus) method, the phase difference is equivalent to the above-mentioned feature. And, when the autofocus is a contrast AF method, the contrast between multiple pixels is equivalent to the above-mentioned feature. In addition, the AF method can be an AF method using directional light, or an AF method using DFD (Depth from defocus).

[0181] Furthermore, the position satisfying the reference refers to the position where the above-mentioned characteristics (e.g., phase difference or contrast, etc.) become the characteristics when focusing. In short, the control unit 46 determines the position focused by the AF function in the reference image as the position satisfying the reference. Then, as Fig.15 As shown, relative to the determined position (the focus position, Fig.15 In this case, for example, the size of the setting area Ar may be automatically set based on the area of ​​the portion in focus in the reference image.

[0182] The image of the portion in focus by the AF function is usually the object of recording in most cases. Based on this, by setting the setting area Ar at the focused position, the image in focus can be easily recorded, thereby improving the convenience for the user.

[0183] In the AF position determination pattern, if the setting area Ar is set at the focused position, the control unit 46 automatically sets another setting area Ar accordingly. Fig.15 As shown in FIG. 1 , another setting area Ar is set at a symmetrical position of the focused position (for example, a point symmetrical position when viewed from the center of the reference image). Thus, when the AF position determination pattern is selected, a plurality of setting areas Ar can be automatically set in the reference area. However, the position where the other setting area Ar is set is not limited to the above position.

[0184] Furthermore, in the AF position determination pattern, when the entire reference image is in focus, the entire reference area can be set as one setting area Ar.

[0185] Furthermore, when the subject is not detected in the reference image and the appropriate setting position of the setting area Ar is not observed in the reference image, the setting area Ar may be set near the center of the reference image.

[0186] In addition, in this embodiment, the focus position in the reference image is cited as an example of the photographic condition, and the setting area Ar is set at a position (i.e., a focused position) where the feature for determining the focus position satisfies the reference in the reference image. Here, the photographic condition may be a condition other than the focus position, for example, exposure and white balance. In addition, the photographic conditions mentioned here exclude shutter speed and ISO sensitivity.

[0187] Here, taking the exposure as one of the photographing conditions as an example, in the reference image, the setting area Ar can be set at a position where the feature (e.g., pixel value or brightness) for determining the exposure satisfies the reference. For example, in the reference image, the setting area Ar can be set at a position where the exposure is adjusted to an appropriate value by the AE (Automatic Exposure) function. Alternatively, in the reference image, the setting area Ar can be set at a position where the exposure is relatively low and at a position where the exposure is relatively high.

[0188] Furthermore, taking white balance as one of the photographic conditions as an example, in the reference image, the setting area Ar can be set at a position where the feature (e.g., chromaticity) for determining the white balance satisfies the reference. For example, in the reference image, the setting area Ar can be set at a position where the white balance is adjusted to an appropriate value by the AWB (Auto White Balance) function. As a specific example, when there are areas where light is irradiated from multiple light sources such as lighting and the sun in the reference image, the setting area Ar can be set for each irradiation area of ​​each light source.

[0189] The user selects any pattern from the multiple configuration patterns described above. Thus, in the setting process, the setting area Ar can be set to reflect the user's ideas or preferences. Furthermore, in the setting process, since the setting area Ar is automatically set, the user does not need to worry about it (setting work). That is, from the perspective of saving the user's trouble and reflecting the user's ideas and setting the setting area Ar in the reference area, the display method of this embodiment improves convenience (ease of use). In addition, the configuration pattern can be Fig.11 The arrangement patterns shown are a combination of a plurality of arrangement patterns. For example, a plurality of setting areas Ar may be set by using both a scene recognition pattern and an AF position determination pattern.

[0190] In addition, the above-mentioned patterns are merely examples, and arrangement patterns other than the above-mentioned patterns may be added, and conversely, a part of the above-mentioned patterns may be omitted.

[0191] Furthermore, in the setting process of the present embodiment, regardless of the type of the selected pattern, when a characteristic part described later exists in the reference image, a candidate Ac of the setting area is automatically set at the position of the characteristic part in the reference area. Then, when the user performs a predetermined operation on the set candidate Ac, the setting area Ar is set at the position of the set candidate Ac.

[0192] Hereinafter, the procedure for setting the candidate Ac in the setting process will be described.

[0193] (About candidate settings)

[0194] In the setting process, the control unit 46 determines whether there is a characteristic part in the reference image based on the characteristics of each pixel of the reference image (for example, pixel value). The characteristic part refers to a part that visually changes in the reference image (excluding the part detected as the subject), for example, a part that moves in the background image and a part that changes in color and brightness. As specific examples, there are parts that flow like a waterfall, parts that light up or go out like a lamp, and parts that switch text and images displayed like an electronic bulletin board.

[0195] In addition, Fig.16 In the reference image shown, for example, the lighting fixture appearing in the upper left portion of the figure corresponds to the characteristic portion.

[0196] When the characteristic part exists in the reference image, the control unit 46 performs the first setting step. The first setting step is a step of automatically setting the candidate Ac of the setting region at the position of the characteristic part in the reference image.

[0197] When the first setting process is performed, in the display process performed thereafter, if Fig.16 As shown, a mark indicating the position of the candidate Ac in the reference image (hereinafter referred to as the candidate mark K) is displayed. The candidate mark K is displayed in a different manner from the mark f indicating the position of the set area Ar. For example, Fig.16 The candidate marker K in is a roughly L-shaped graphic object representing a quadrilateral of a rectangular area surrounding the feature part. There is no particular limitation on the type and shape of the candidate marker K. For example, it can be an arrow or a pointer in the shape of a finger, or a graphic object composed of a circle or a star. Alternatively, the difference in shading (difference in brightness) between the candidate Ac in the reference image and its surroundings can be set as the candidate marker K.

[0198] If the user performs an operation such as clicking the candidate mark K during the display process, this is used as a trigger to implement the second setting process. The second setting process is a process in which the control unit 46 sets the setting area Ar at the setting position of the candidate Ac where the operation is performed according to the user's operation (e.g., clicking operation, etc.) on the candidate Ac. By implementing the second setting process, Fig.17 As shown in FIG. 1 , a new setting area Ar is set in the area of ​​the image of the characteristic part.

[0199] As described above, in this embodiment, the candidate Ac of the set area can be automatically set to a characteristic part that is not recognized as a subject in the reference image but has a visual change. And, by displaying the candidate mark K indicating the position of the set candidate Ac in the reference image, the user can easily confirm the candidate Ac. And, by the user performing a predetermined operation on the candidate Ac, the set area Ar can be set at the position of the candidate Ac. Thus, in the reference image, the set area Ar can be set for the characteristic part that the user is interested in. And, if the set area Ar set in this way is selected as the recording area, the image of the characteristic part can be recorded.

[0200] [About the display process]

[0201] The following describes in detail the display process in the image display flow.

[0202] When the setting area Ar is set in the setting process, the control unit 46 starts the display process. Thereafter, the control unit 46 continues the display process until the recording process is completed (strictly speaking, until the shooting is completed).

[0203] The display step is a step of displaying the recorded image, the reference image, and the marks f indicating the positions of the plurality of set areas Ar in the reference image on a display. Fig.18 As shown, the mark f is displayed in the reference image for each set area Ar. Thus, the user can reliably grasp the position of each set area Ar in the reference image.

[0204] Furthermore, in the display process, Fig.18 As shown, each image (hereinafter referred to as a standby image) of the setting area Ar excluding the recording area Ap among the plurality of setting areas Ar can be displayed.

[0205] Furthermore, in the display step when the aforementioned characteristic part exists in the reference image, the candidate mark K indicating the position of the characteristic part is displayed in the reference image together with the mark f.

[0206] An example of the display process is described below. The recorded image, the reference image, and the standby image can be simultaneously displayed on the screen of the rear display 28 (hereinafter referred to as the display screen). Fig.18 As shown in FIG. 1 , the display screen is divided into a main screen G1 , a sub screen G2 , and an information display screen G3 . In this case, the standby image can be displayed on the sub screen G2 .

[0207] The main screen G1 occupies a relatively wide area (for example, more than half of the display screen) on the display screen and displays the reference image. The aspect ratio of the main screen G1 is preferably set to the same value as the aspect ratio of the reference image.

[0208] The sub-screens G2 are smaller than the main screen G1 and are arranged in a row along the vertical or horizontal direction of the display screen. Each sub-screen G2 displays a recorded image or a standby image. It is preferred that the aspect ratio of each sub-screen G2 is set to the same value as the aspect ratio of the recorded image and the standby image.

[0209] The information display screen G3 is a screen that displays text information in addition to images, for example, Fig.18 As shown, suggestions related to image recording are displayed.

[0210] In addition, the layout of the display screen (for example, the size and arrangement position of the main screen G1 and the sub-screen G2) is not limited to Fig.18 The method shown can be freely designed according to the specifications of the device or the user's preference.

[0211] In addition, in the main screen G1, together with the reference image, a mark f is displayed for each of the plurality of setting areas Ar set in the setting process. In the display process, the number of marks f displayed in the main screen G1 (display number) is variable, and the maximum number of marks f that can be displayed (display upper limit number) can be displayed simultaneously.

[0212] Furthermore, in order to make it easier for the user to see the mark f, it is more preferable to highlight the image within the mark f in the reference image. Fig.18 As shown, in the reference image, the brightness of the image other than the image within the mark f (ie, within the boundary of the set area Ar) may be reduced or grayed out.

[0213] Furthermore, in the display process, the mark f indicating the position of the recording area Ap and the mark f indicating the position of the other setting area Ar (hereinafter also referred to as the standby area) are displayed in different ways. At this time, the mark f indicating the position of the recording area Ap is preferably displayed in an emphasized manner. For example, the color of the frame line can be set to a relatively conspicuous color such as red or fluorescent color, or the frame line can be set to be thicker or flashing. Furthermore, the image (i.e., the recorded image) itself in the recording area Ap can be displayed in an emphasized manner.

[0214] Furthermore, when there is a moving subject in the reference image, the moving area Am set for the subject moves to follow the subject. Fig. 9 As shown in FIG. 1 , the mark f indicating the position of the moving area Am is displayed in a manner different from the mark f indicating the position of the set area Ar other than the moving area Am. Thus, the moving area Am can be easily distinguished from the set area Ar other than the moving area Am, and the position of the moving area Am can be easily observed in the reference image.

[0215] In addition, if the subject in the moving area Am moves in the depth direction (i.e., the direction of approaching and moving away from the imaging device 10), the size of the subject in the reference image changes. With this, the ratio of the size of the subject to the size of the moving area Am (hereinafter referred to as the subject ratio) changes. In this case, the control unit 46 uses the electronic zoom function to change the size of the moving area Am according to the change of the subject ratio, and adjusts the subject ratio to be constant.

[0216] Furthermore, as the subject moves, the moving area Am may move to the end of the reference area (i.e., the outer edge of the field of view) or exceed the reference area. In this case, the control unit 46 preferably displays a warning message on the information display screen G3 or outputs a warning sound to notify the user of the above situation.

[0217] Furthermore, if the reference image (i.e., the extracted image extracted from the image of the total field angle) is changed by electronic hand shake correction, the relative position of each set area Ar with respect to the reference area changes (refer to Figure 4 In this case, the control unit 46 calculates the deviation of the reference image based on the electronic hand shake correction, and adjusts the position of each set area Ar based on the calculated deviation. Thus, the position of each set area Ar relative to the reference area is maintained before and after the implementation of the electronic hand shake correction.

[0218] In addition, it is preferable that the function of adjusting the position of the setting area Ar in conjunction with the electronic image stabilization be freely switched between on and off states in accordance with a user's operation.

[0219] Furthermore, in the display process, Fig.18 As shown, the identification information of each mark f is displayed for each mark f. The identification information of each mark f is fixed during the implementation of the display process, and even if the display number of the mark f changes during the display process, it is maintained without change before and after. For example, Fig.18 In the example, when the display of the mark f with the identification number "3" is stopped, the identification numbers of the other marks f (i.e., "1", "2", and "4") do not change. In this way, by fixing the identification numbers of the marks f during the implementation of the display process, for example, even if the number of displayed marks f changes, it is possible to easily grasp which set area Ar exists where.

[0220] In the plurality of sub-screens G2, a plurality of setting images including recording images and standby images are displayed. Specifically, the corresponding setting images are displayed one by one on the plurality of sub-screens G2. Here, if the aspect ratio of each sub-screen G2 is the same as the aspect ratio of the corresponding setting image, the setting image can be well accommodated in the screen and displayed.

[0221] Furthermore, among the multiple setting images displayed, the recorded image is preferably displayed with emphasis. For example, the color of the frame, the thickness of the frame line or the type of line, or the presence or absence of lighting are preferably different between the sub-screen G2 displaying the recorded image and the sub-screen G2 displaying other setting images. Thus, a user who observes the multiple setting images on the display can easily distinguish which image is the recorded image.

[0222] And, if Fig.18 As shown, in the sub-screen G2 ( Fig.18 In the second sub-screen from the top, information indicating the image of the moving area Am (for example, text information such as "moving" etc.) is displayed.

[0223] In this embodiment, a plurality of execution modes of the display process are prepared, and the display method of the image in the display process changes for each mode. That is, the display process is executed according to the mode designated by the user.

[0224] For example, if the user specifies the first mode, the first display process is performed as the display process. Fig.18 As shown in FIG. 1 , in the first display process, the reference image and the mark f of each set area Ar are displayed on the main screen G1, and the recorded image and the standby image are displayed on the sub-screen G2. On the other hand, if the user specifies the second mode, the second display process is implemented as the display process. Fig.19 As shown, in the second display step, the recorded image and the standby image are displayed, and the reference image and the mark f for each set area Ar are not displayed.

[0225] In addition, in the first display step, the recorded image, the standby image, the reference image, and the mark f of each set area Ar do not need to be displayed on the same display (for example, the rear display 28). Fig. 20 As shown, the image displayed in the main screen G1 and the image displayed in the sub-screen G2 can be displayed on different displays. For example, the image displayed in the main screen G1 can be displayed on the rear display 28, and the image displayed in the sub-screen G2 can be displayed on the electronic viewfinder 29 or an external display (not shown) connected to the photographing device 10.

[0226] Thus, in this embodiment, the display process includes a first display process and a second display process, and of the two display processes, the process specified by the user is implemented. Thus, the display image can be switched in accordance with the needs of the user. For example, when the user wants to confirm the reference image and the mark f, the first display process can be implemented to display the reference image and the mark f. On the other hand, when the user wants to confirm only the recorded image and the standby image (i.e., the set image), the second display process can be implemented to display only the recorded image and the standby image.

[0227] Furthermore, the execution mode of the display process, that is, which of the first display process and the second display process is to be executed, is preferably freely switched during the execution of the display process according to a switching operation by the user. Figure 2 The touch panel 36 or the operation button 30 shown is used to switch which display process is to be adopted.

[0228] In addition, in the second display step, the recorded image and other set images can be displayed on the same display. Fig.19 As shown, the recorded image can be displayed on the main screen G1 of the display screen, and the standby image can be displayed on the sub screen G2.

[0229] Alternatively, in the second display step, the recorded image and the other set image may be displayed on different displays. Fig. 20 As shown, the recorded image can be displayed on the rear display 28, and other set images can be displayed on the electronic viewfinder 29 or an external display (not shown) connected to the photographing device 10.

[0230] On the information display screen G3, for example, suggestion information described later is displayed.

[0231] Furthermore, the information display screen G3 can display advice and warning messages for the user regarding photography, etc. For example, when the exposure or ISO sensitivity needs to be adjusted during the recording process, such a message can be displayed.

[0232] Furthermore, the information display screen G3 can display information other than the above, such as the shooting mode currently used, current values ​​of shooting conditions such as resolution and ISO sensitivity, and the on / off status of various functions such as electronic zoom.

[0233] (About display of setting images based on priority)

[0234] In the display process, the number of sub-screens G2 is determined based on at least one of the resolution and aspect ratio of the recorded image (hereinafter referred to as "at least one of the items"). That is, the number of set images displayed on the sub-screen G2 is determined based on at least one of the items.

[0235] Similarly, the size of each sub-screen G2 is also determined based on at least one of the items. That is, the display size of the image is set to a size corresponding to at least one of the items.

[0236] Here, at least one of the items is adjusted according to the recording format (movie format) of the recorded image. Therefore, the display number (strictly speaking, the maximum display number) and display size of the set image in the display process are the number and size corresponding to the recording format of the recorded image.

[0237] In this embodiment, the number of setting images to be displayed and the display size are determined based on at least one of the items, so that each of the plurality of setting images including the recorded image can be well displayed. For example, each setting image can be well accommodated in the sub-screen G2 with an appropriate resolution and displayed.

[0238] In addition, in the present embodiment, the aspect ratio of the recorded image is determined according to the recording format of the recorded image, but the aspect ratio of the recorded image may be arbitrarily set regardless of the recording format. In this case, as the aspect ratio of the recorded image is adjusted, the number and size of the sub-screens G2 change according to the adjusted aspect ratio.

[0239] Furthermore, in the present embodiment, the number and size of the set images displayed in the display process (i.e., the number and size of the sub-screens G2) are determined based on at least one of the items. However, this is not limiting, and either the number or size of the set images can be determined based on at least one of the items.

[0240] In addition, the number of display images in the display step may be determined based on at least one item. For example, the user may set or change the number of display images.

[0241] In this embodiment, the number of display of the setting images in the display process is determined according to the resolution and aspect ratio of the recorded image as described above. Here, in the case of the first display process, among the setting images displayed, one is a recorded image and the rest are standby images. Therefore, the setting value related to the display number of the standby images, that is, the maximum display number, is determined according to the resolution and aspect ratio of the recorded image, and specifically, is the number less than the number of the sub-screens G2 by 1.

[0242] In addition, Fig.21In the case shown, the maximum number of standby images displayed is three.

[0243] On the other hand, the number of set areas Ar in the reference area changes according to the subject in the reference image, and accordingly, the number of set areas Ar other than the recording area Ap, that is, the number of the plurality of waiting areas, changes.

[0244] In addition, Fig.21 In the case shown, the number of standby areas is six.

[0245] When the number of the plurality of waiting areas reaches (strictly speaking, exceeds) the maximum display number of the waiting images, the control unit 46 Fig. 22 The process shown in the figure implements the display process. Fig. 22 The flow of the display process shown in FIG. 1 is described. When the number of the plurality of waiting areas reaches the maximum display number of the waiting images, the control unit 46 implements the determination process ( S021 ).

[0246] In the determination process, the control unit 46 determines the priority for each of the plurality of standby areas. To explain this step, first, the control unit 46 detects a subject in the standby image for each of the plurality of standby images (S101). Next, the control unit 46 acquires information related to the detected subject in the standby image (S102). In addition, at this time, the control unit 46 may acquire performance information as a recording area described later instead of the information related to the subject in the standby image.

[0247] The information related to the subject may include, for example, the time or number of times the subject appears in the reference image during the shooting period, and the size (display size) of the subject in the reference image. This information can be obtained by analyzing the reference image and measuring or counting by the control unit 46 during the shooting period.

[0248] Furthermore, when the subject is a person, the face of the person can be identified through pattern matching and other techniques, the importance of the person whose face is identified can be determined, and the determination result can be set as information related to the subject. Regarding the importance of the person, for example, the attributes of the person whose face is identified (for example, family members, relatives, acquaintances, or other people) can be determined and obtained from the determination result. Furthermore, when there are multiple subjects detected, the priority can also be determined by the category of the subject (for example, people, animals, objects, or scenery, etc.). Furthermore, the priority can also be determined starting from the subject that is detected earlier.

[0249] In addition, it is obvious that information other than the above-mentioned information may be considered as the information related to the subject. In the determination step, it is desirable that the user freely selects which information to use as the information related to the subject.

[0250] Thereafter, in a determination step, the control unit 46 determines the priority levels of the plurality of waiting areas based on information on the detected subject ( S103 ).

[0251] After the determination process is performed, the control unit 46 selects N standby areas from the plurality of standby areas based on the priority (S022). Here, the number N of standby areas selected is the maximum number of standby images to be displayed, and is a number determined according to the number of sub-screens G2 (for example, a number that is less than the number of sub-screens G2 by only 1). In step S022, the control unit 46 selects N standby areas in order from the one with the highest priority.

[0252] Then, the control unit 46 displays the images of the selected N standby areas (standby images) on the sub-screen G2 (S023). Fig.21 In the case of the arrangement along the vertical direction of the display screen as shown, it is desirable to display the standby image of the standby area with a higher priority on the upper sub-screen G2.

[0253] As described above, in this embodiment, when the number of the plurality of standby areas reaches (exceeds) the maximum display number of standby images, N standby images selected according to the priority are displayed. Thus, it is possible to appropriately display high-priority standby images while responding to the limitations in screen display.

[0254] In addition, the method for determining the priority is not limited to the method based on the information related to the subject in the standby image. For example, the priority can be determined based on the performance information of the recording area. To explain in more detail, for each of the multiple standby areas, the number of times it has been selected as a recording area from the start of shooting to the current time point, the time it has been in the recording area, or the time elapsed from the time point when it was selected as a recording area immediately before is determined. This information is equivalent to information related to the performance of being selected as a recording area in the past.

[0255] Then, the priority of each of the plurality of waiting areas may be determined based on the information related to the above performance. For example, when the information related to the performance is the number of times or time when the waiting area is selected as a recording area, the greater the number of times or time, the higher the priority may be given.

[0256] And, in Fig. 22 In the flow of the display process shown in FIG. 1 , N standby areas are selected based on priority, and only the images of the selected standby areas are displayed, but the present invention is not limited thereto. For example, the upper N standby areas and the lower standby areas may be selected in descending order of priority. In this case, the standby images of the upper N standby areas may be displayed first, and then the images of the selected standby areas may be displayed. Fig.23As shown, the standby image of the lower standby area is displayed by scrolling or switching the display screen.

[0257] Furthermore, each of the plurality of standby images may be displayed in a size corresponding to the priority. Fig.24 As shown, the standby image of the high priority standby area can be displayed on the normal size sub-screen G2, and the standby image of the low priority standby area can be displayed on the reduced size sub-screen G2x. In this way, each of the plurality of standby images can be displayed, and the high priority standby area can be displayed more easily.

[0258] [About the control process]

[0259] The control process in the image display process is described. The control process is a process for executing a control process for controlling the change of the reference area, and is performed at least after the reference area is determined. That is, the control process is a process for controlling the change of the reference area after the time point of receiving the recording start instruction from the user.

[0260] In the control process, the control unit 46 locks the zoom drive unit 21 and stops the movement of the zoom optical device 18. Thus, in the recording process, the optical zoom function is limited, the reference area is fixed, and the viewing angle at that time is maintained.

[0261] By executing the above control process, the relative position of each setting area Ar with respect to the reference area can be prevented from changing during the recording process. As a result, it is possible to avoid a situation where each setting area Ar deviates from the reference area and each setting image such as a recording image cannot be properly captured.

[0262] In addition, the control process implemented to suppress the change of the reference area is not limited to the above-mentioned process, and may be, for example, a process for notifying the proposed information related to the change of the reference area. As a specific example, a message prohibiting the optical zoom may be displayed on the information display screen G3 or reproduced by sound or the like. Fig.18 As shown, a message prohibiting the movement of the photographing device 10 (for example, panning) or a message recommending the use of a tripod or other head to fix the photographing device 10 is also displayed.

[0263] Furthermore, the control process may be implemented before determining the reference region (ie, before receiving the recording start instruction), or in the setting process, for example. In this case, the reference region can be prevented from changing during the setting of a plurality of setting regions Ar in the reference region.

[0264] [About process changes]

[0265] During the recording process, if a predetermined condition is satisfied, the control unit 46 performs a change process. The change process is a process for performing a change process related to a change in the number of images (i.e., set images) displayed in the set area Ar. By performing the change process, the number of set images displayed can be dynamically increased or decreased during the recording process.

[0266] Below, reference Fig.25 The process shown in the figure will be used to describe the process of processing during the recording process. Figure 26 to Figure 31 In the figure, for the convenience of illustration, the graying out of images other than the image within the mark f is omitted.

[0267] During the recording process, the control unit 46 monitors the reference image and determines whether the subject in the reference image increases (S031). If the subject in the reference image increases, the control unit 46 implements a change process (S032). In the change process, a change process is performed to increase the display number of the set image.

[0268] In the change process in step S032, the setting area Ar is added according to the increase of the subject in the reference image, and the setting image of the added setting area Ar is displayed on the new display. As a result, the number of setting images displayed increases. Fig.26 and 27 As shown, when the number of subjects in the reference image increases from three to four, the number of displayed images (ie, the number of images displayed in the sub-screen G2) is increased from three to four.

[0269] When the setting area Ar is added as the subject in the reference image increases, the number of marks f in the reference image displayed on the main screen G1 increases by the same amount as the setting area Ar added in a linked manner (see FIG. Fig. 27 ).

[0270] Furthermore, in the above-mentioned embodiment, the setting area Ar is set at the position of the new subject in the reference image, but the present invention is not limited thereto. For example, a candidate Ac of the setting area may be set instead of the setting area Ar. In this case, a mark indicating the position of the candidate Ac (specifically, the candidate mark K mentioned above) is displayed in the reference image. Then, if the user clicks the candidate mark K or performs an operation, it is used as a trigger to add the setting area Ar at the position of the candidate Ac. As a result, the setting image of the added setting area Ar is newly displayed, and the number of setting images displayed increases.

[0271] Furthermore, instead of setting the setting area Ar or its candidate Ac at the position of the new subject in the reference image, the suggestion information related to the addition of the setting area Ar (i.e., the change of the display number of the setting image) may be output. For example, the setting area Ar may be added at the position of the new subject and a message suggesting the increase of the display number of the setting image may be displayed on the information display screen G3 or reproduced by sound or the like. In this way, as the change processing, the processing of outputting the suggestion information related to the change of the display number of the setting image may be performed.

[0272] exist Fig.25 In the process shown in FIG. 1 , the control unit 46 monitors the reference image during the recording process and determines whether the moving area Am in the reference area overlaps with other set areas Ar ( S033 ). Fig.28 As shown in FIG. 1 , when the moving area Am overlaps with the other set area Ar, or just before the overlap, the control unit 46 outputs a warning message or a warning sound (S034). This is because when the moving area Am overlaps with the other set area Ar, there is a situation where the subject in one area appears behind the subject in the other area and cannot be imaged.

[0273] By outputting a warning message, etc., the user can be informed that the moving area Am overlaps with other set areas Ar or is about to overlap. The user who receives the warning can take appropriate measures (for example, resetting the set area Ar, etc.). As one of the measures, when the user makes a change request through screen operation, etc. (S035), the control unit 46 implements a change process (S036). In this change process, a change process of reducing the number of display of the set image is performed.

[0274] In the change process in step S036, the overlapping moving area Am and other setting areas Ar are updated to one setting area Ar, and the setting image of the updated setting area Ar is displayed on the display. Fig.28 The images of the moving area Am and the images of other set areas Ar are displayed separately as shown in FIG. Fig.29 In this way, the two images are combined into one image including the respective objects. As a result, the number of set areas Ar is reduced by one, and the number of set images to be displayed is also reduced by one.

[0275] In addition, the size and position of the updated setting area Ar are preferably set to accommodate the subjects of the above two images. Furthermore, the identification information of the mark f representing the updated setting area Ar may be the same as the identification information of the mark f representing any one of the two areas before the update (i.e., the moving area Am and the other setting area Ar). Alternatively, a new identification number may be assigned to the updated setting area Ar.

[0276] exist Fig.25 In the process shown, the control unit 46 periodically determines whether each of the plurality of setting areas Ar satisfies the termination condition (S037) during the recording process. The termination condition is a criterion for determining whether the display of the image (setting image) of the setting area Ar should be terminated, and the following conditions (1) to (3) can be cited as examples.

[0277] Termination condition (1): at least a portion of the set area Ar exceeds the reference area due to optical zoom or the like.

[0278] Termination condition (2): For a certain period of time or longer, there is no change in the setting image of the setting area Ar or there is no subject in the setting image.

[0279] Stop condition (3): The set area Ar is not selected as the recording area for a certain period of time or longer.

[0280] In addition, the termination conditions are not limited to the above-mentioned conditions (1) to (3), and conditions other than the above-mentioned conditions may be included in the termination conditions.

[0281] Then, when there is a setting area Ar that satisfies any one of the above-mentioned stop conditions (1) to (3), the control unit 46 implements a change process (S038). In this change process, a change process of reducing the number of display settings is executed.

[0282] In the change process in step S038, the display of the setting image of the setting area Ar that satisfies the suspension condition is suspended. Fig.26 In the case where the setting area Ar of the identification information of the flag f is "2" satisfies the termination condition, such as Fig.30 As shown in FIG. 1 , the display of the setting image of the setting area Ar is stopped (disappears from the sub-screen G2 ). As a result, the number of displayed setting images decreases by one.

[0283] In addition, regarding the setting area Ar where the display of the setting image is suspended (hereinafter referred to as the suspension area), as shown in FIG. Fig.30 As shown, in the main screen G1, the display of the mark f indicating the position of the pause area is also paused.

[0284] That is, instead of immediately stopping the display of the setting image of the setting area Ar that meets the stop condition, it is also possible to output the suggestion information related to the display stop of the setting image (in other words, the change of the display quantity of the setting image). For example, a message recommending the display stop of the setting image of the setting area Ar that meets the stop condition may be displayed on the information display screen G3, or reproduced by sound or the like. During this period, another image (an image different from the setting image) may be displayed in the sub-screen G2 instead of the setting image of the setting area Ar that meets the stop condition.

[0285] If the display of the pause image setting image is paused, the pause area is stored in the internal memory 50 etc. in the photographing device 10 during the recording process. The pause area may be stored until the shooting is completed or the power of the photographing device 10 is turned off.

[0286] Furthermore, if the display of the set image of the stop area is stopped, Fig.30 As shown in FIG. 1 , an operation icon such as a return button Bt9 is displayed in the main screen G1. The user can, for example, click the return button Bt9 to instruct the setting image of the pause area to be redisplayed. If there is a redisplay instruction from the user (S039), the control unit 46 implements a change process (S040). In this change process, based on the redisplay instruction from the user, a change process is executed to redisplay the setting image of the pause area stored in the storage.

[0287] By the change processing in step S040 , the image of the pause area is re-displayed on the sub-screen G2 , and the number of displayed set images returns to the number before the display of the set images in the pause area was paused.

[0288] In addition, the setting image of the pause area that can be redisplayed is not limited to the image of the immediately previous pause area (the setting area whose display of the setting image was most recently suspended). For example, the images of the pause areas of the past several times during the current shooting period may be redisplayed. In this case, the past several pause areas are stored in the photographing device 10, and the user specifies which setting image of the pause area is to be redisplayed and instructs the redisplay of the image.

[0289] exist Fig.25 In the flow shown, the control unit 46 monitors the reference image during the recording process and determines whether the number of subjects in the reference area exceeds the upper limit of the number of set areas Ar (S041). Then, when the number of subjects exceeds the upper limit of the number of set areas Ar, the control unit 46 implements a change process (S042). In this change process, a change process related to a reduction in the number of displayed set images is performed.

[0290] In the change process in step S042, the fact that the number of subjects exceeds the upper limit of the number of setting areas Ar and a message recommending to reduce the number of displaying setting images is displayed on the information display screen G3 or reproduced by sound, etc. The user takes this as an opportunity to modify the setting of the setting area Ar, for example, delete unnecessary setting areas Ar such as the setting area Ar that has not been selected as the recording area for the longest period. As a result, the number of displaying setting images is reduced by only the number of setting areas Ar deleted by the user.

[0291] The series of steps S031 to S042 described above are repeatedly performed during the recording process, and are terminated at the end of the recording process ( S043 ).

[0292] As described above, in this embodiment, the change process is implemented based on the change in the reference image (for example, the addition of a subject, etc.) and the establishment of the termination condition, and the number of displayed set images is appropriately changed. As a result, the user can easily confirm the number of set images corresponding to the current situation, and can omit the confirmation of the image of the set area Ar, etc., which has a low possibility of becoming a recording area. As a result, the convenience for the user is improved.

[0293] <<Other implementation methods>>

[0294] The embodiment described above is a specific example given to explain the display method of the present invention in an easy-to-understand manner. It is just an example, and other embodiments are also conceivable.

[0295] In the above-mentioned embodiment, the photographing device 10 is used as a display device to display various images, but the present invention is not limited thereto. For example, the display device may be composed of the photographing device 10, a camera controller connected to the photographing device 10 by wire or wirelessly, and an external display. In this case, the camera controller functions as the control unit 46 to make the photographing device 10 capture a reference image and to make the external display display various images. In addition, the camera controller may also set the setting area Ar, select and reselect (switch) the recording area, and record the recording image.

[0296] Furthermore, in the above-mentioned embodiment, in the control process in the image display process, it is set to perform a control process for suppressing the change of the reference area. However, it is not limited to this, and as a control process related to the change of the reference area, a process for urging the change of the reference area may also be performed. In this case, the reference area can be adjusted according to the change in the reference image (for example, the movement of the subject in the reference image, etc.).

[0297] To explain in more detail, for example, in the recording process, the recorded image is zoomed and enlarged by electronic zoom. At this time, if the zoom ratio exceeds a predetermined value, the resolution of the recorded image becomes below a certain value, and it may be difficult to obtain the desired image quality. In this case, if Fig.31 As shown, optical zoom may be performed instead of electronic zoom, and information prompting change of the reference area may be displayed on the information display screen G3. Alternatively, for example, the control unit 46 may move the zoom drive unit and forcibly perform optical zoom so that the resolution becomes equal to or greater than a predetermined value, thereby automatically changing the reference area.

[0298] If the reference area changes due to optical zoom, the relative position of each set area Ar with respect to the reference area changes. In this case, the control unit 46 preferably calculates the amount of change in the reference area, and adjusts the position and size of each set area Ar based on the calculated amount of change. Thus, the relative position of each set area Ar with respect to the reference area is maintained before and after the zoom is implemented.

[0299] Furthermore, during the recording process, the moving area Am sometimes follows the movement of the subject and reaches the vicinity of the end of the reference area, or at least a part of the moving area exceeds the reference area. In this case, information urging the photographing device 10 to move so that the moved moving area Am is included in the reference area (i.e., the reference image is changed) can be displayed on the information display screen G3.

[0300] Here, in the recording process, when the user changes the reference area by moving the photographing device 10, a known detection sensor such as a gyro sensor provided in the photographing device 10 can detect the movement (for example, displacement) of the photographing device 10. Then, the position and size of each set area Ar are preferably adjusted based on the movement of the photographing device 10 detected by the control unit 46. Thus, the relative position of each set area Ar with respect to the reference area is adjusted according to the changed reference area. At this time, the set area Ar that exceeds the reference area due to the change of the reference area can be automatically deleted.

[0301] Furthermore, in the above-mentioned embodiment, the recorded image is an animation, that is, a collection of multiple frame images continuously captured at a constant frame rate. However, the recorded image is not limited to an animation, and may also be a static image.

[0302] For example, the control unit 46 displays the reference image as a live view image on the main screen G1, and sets a plurality of setting areas Ar in the reference area. The user selects a recording area Ap from the plurality of setting areas Ar, and the control unit 46 displays the plurality of setting images including the recording image on the sub-screen G2. Then, if the user operates the release button 26 or the like to give a recording instruction, the control unit 46 can record the recording image as a still image on the recording medium.

[0303] Furthermore, in the above-mentioned embodiment, when a subject is detected in the reference image, the setting area Ar is automatically set at the position of the detected subject. However, the setting area Ar is not limited to being immediately set at the position of the detected subject, and a candidate Ac of the setting area may be first set at the position of the detected subject. In this case, a candidate mark K is displayed at the position of the candidate Ac, and when the user performs a predetermined operation on the candidate mark K, the setting area Ar can be set at the position of the candidate Ac.

[0304] Furthermore, in the above-mentioned embodiment, regarding the setting area Ar that satisfies the termination condition, the display of the setting image is suspended, and on the other hand, it is stored in the photographing device 10 during the implementation of the recording process. However, this is not limited to this, and the setting area Ar that satisfies the termination condition may be deleted, and the image of the area may not be stored. In this case, it is preferable that the suggestion information related to the deletion of the setting area Ar that satisfies the termination condition is displayed on the information display screen G3 before deleting the setting area Ar that satisfies the termination condition. Then, a screen (not shown) for selecting whether to delete the setting area Ar may be displayed separately, and after confirming the user's opinion on whether to delete, the setting area Ar that satisfies the termination condition is deleted.

[0305] Furthermore, in the above-mentioned embodiment, the photographing device is a digital camera, but it may also be a video camera, a mobile phone with an imaging optical system, a smart phone, a tablet terminal, or other mobile terminal. Furthermore, the imaging lens may also be a lens unit of the imaging optical system external to the mobile terminal.

[0306] Explanation of symbols

[0307] 10-photographic device, 12-photographic device body, 13-mount, 14-photographic lens, 17-optical unit, 18, 19-optical device, 20-aperture, 21-zoom drive unit, 22-focus drive unit, 23-aperture drive unit, 26-release button, 28-display, 29-electronic viewfinder, 30-operation button, 36-touch panel, 38-shutter, 40-imaging element, 42-pixel, 44-analog signal processing circuit, 46-control unit, 47-controller, 48-image processing unit, 50-internal memory, 52-card slot, 54- Memory card, 56-buffer memory, 61-camera controller, 62-external display, 101-1st UI screen, 102-2nd UI screen, 103-3rd UI screen, 104-4th UI screen, 105-input screen, Ac-candidate, Am-moving area, Ap-recording area, Ar-setting area, As, At-shooting area, Bt1~Bt8-buttons, Bt9-return button, f, fx-marks, G1-main screen, G2, G2x-sub screens, G3-information display screen, K-candidate mark, L1-optical axis, T-table data.

Claims

1. A display method for displaying an image captured by a photographing device, the display method comprising: a setting step of automatically setting a plurality of setting areas or candidates for the setting areas in a reference area as a photographing area of ​​a reference image; A selection step of selecting a recording area as an area of ​​a recording image to be recorded from the plurality of set areas; and a switching step of reselecting the recording area from the plurality of setting areas and switching the recording area after the selection step is performed, The setting process comprises: A first setting step of determining, in the reference region, whether there is a portion causing visual changes, i.e., a characteristic portion, in the reference image based on the characteristics of each pixel of the reference image, and automatically setting the candidate at the position of the characteristic portion in the reference image if the characteristic portion exists in the reference image; and The second setting step is to set the setting area at a position where the candidate is set, based on a user's operation on the candidate.

2. The display method according to claim 1, wherein: In the setting step, a photographing scene of the reference image is recognized based on the reference image, and the setting area or the candidate is automatically set at a position corresponding to the recognized photographing scene in the reference area.

3. The display method according to claim 1, wherein: In the setting step, the setting area is automatically set at a position in the reference image where a feature for determining a photographing condition of the photographing device satisfies a reference.

4. The display method according to claim 3, wherein: The photographing condition is at least one of a focus position, an exposure amount, and a white balance in the reference image.

5. The display method according to claim 1, wherein: In the setting step, the setting area or the candidate is automatically set based on the position information of the setting area stored in advance.

6. The display method according to claim 5, wherein: The position information includes at least one of the position information of the setting area set in the setting process performed in the past and the position information of the setting area determined based on a user operation.

7. The display method according to claim 1, further comprising: The adjusting step is to adjust at least one of the position and the size of the setting area when the size or the position of the reference area changes.

8. A display method for displaying an image captured by a photographing device, the display method comprising: A setting step of setting a plurality of setting areas or candidates for the setting areas in a reference area as a photographing area of ​​a reference image; A selecting step of selecting a recording area as an area of ​​a recording image to be recorded from the plurality of set areas; a switching step of reselecting the recording area from the plurality of setting areas and switching the recording area after the selection step is performed; A recording step of recording a dynamic image of the recorded image; A display step of displaying a set image as an image of the set area; and a changing step of executing a change process related to a change in the display number of the set image during the implementation of the recording step, The setting process comprises: A first setting step of determining, in the reference region, whether there is a portion causing visual changes, i.e., a characteristic portion, in the reference image based on the characteristics of each pixel of the reference image, and automatically setting the candidate at the position of the characteristic portion in the reference image if the characteristic portion exists in the reference image; and The second setting step is to set the setting area at a position where the candidate is set, based on a user's operation on the candidate.

9. The display method according to claim 8, wherein: The change processing includes a process of adding the set area and increasing the display number according to an increase in the subject in the reference image.

10. The display method according to claim 8 or 9, wherein: The change process includes a process of stopping display of the set image of the set area that satisfies a stop condition and reducing the display quantity.

11. The display method according to claim 8 or 9, wherein: The pause area of ​​the setting area where display of the setting image is paused is stored during the recording step. The change process includes a process of redisplaying the stored setting image of the pause area based on a redisplay instruction from a user.

12. The display method according to claim 8 or 9, wherein: The change processing includes a process of outputting proposal information related to the change of the display quantity.

13. The display method according to claim 8 or 9, wherein: When the number of subjects in the reference image exceeds the upper limit value of the number of the set areas, the change process related to the reduction of the display number is executed in the change step.

Images