Representation method

Abstract

To improve convenience to a user by a display method for video capable of cutting a part of a photographic video and changing over the cut video.SOLUTION: A display method for displaying a video photographed by an imaging device comprises: a setting process of setting a plurality of set regions in a reference region as a photographic region of a reference video; a selecting process of selecting a recording region as a region of a recorded video out of the plurality of set regions; a changing-over process of selecting the recording region out of the plurality of set regions and changing over the recording region after the selecting process; a display process of displaying a plurality of set videos as videos of the plurality of set regions; and a control process of executing control processing related to change of the reference region at least after determining the reference region. The control processing includes processing to maintain a state in which an optical zooming function of the imaging device is restricted throughout the recording process.SELECTED DRAWING: Figure 18

Description

Technical Field

[0001] One embodiment of the present invention relates to a method for displaying video.

Background Art

[0002] A technique for extracting and storing the video of any one of a plurality of subjects in a video captured by a single imaging device has already been developed. As an example, for instance, the technique described in Patent Document 1 can be cited. In the technique described in Patent Document 1, a plurality of different video regions are set within the imaging range of the imaging device, and the video of one of those regions is recorded. Also, in the technique described in Patent Document 1, the region of the video to be recorded can be changed (switched) among a plurality of regions. Thereby, for example, while keeping the imaging device fixed, it is possible to obtain a zoomed-in video for each of a plurality of subjects in the captured video.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One embodiment of the present invention has been made in view of the above circumstances, and aims to improve the convenience for the user in a video display method of extracting a part of a captured video and being able to switch the extracted video.

Means for Solving the Problems

[0005] To achieve the above objective, one embodiment of the present invention is a display method for displaying an image captured by a camera, comprising: a setting step of setting a plurality of setting areas within a reference area which is the shooting area of ​​a reference image; a selection step of selecting a recording area which is the area of ​​the recorded image to be recorded from among the plurality of setting areas; a switching step of switching the recording area by re-selecting the recording area from among the plurality of setting areas after the selection step has been performed; and a display step of displaying the recorded image, a reference image, and markers indicating the positions of each of the plurality of setting areas within the reference image.

[0006] Furthermore, the marker may include the boundary of the set area, and in the display process, the image within the boundary may be highlighted from the reference image.

[0007] Furthermore, at the start of the setup process, the indicators may be displayed in a state where the respective setup areas do not overlap.

[0008] Furthermore, in the display process, a sign indicating the position of the recording area within the reference video and a sign indicating the position of the standby area, which is a setting area other than the recording area, may be displayed in different manners.

[0009] Furthermore, in the display process, the indicators showing the position of the moving area, which is a set area that moves in accordance with the moving subject, may be displayed in a different manner than the indicators showing the positions of set areas other than the moving area.

[0010] Furthermore, in the display process, the recorded video and the reference video may be displayed on different displays.

[0011] Furthermore, the display process includes a first display process that displays the recorded video, the reference video, and the sign, and a second display process that displays the recorded video but does not display the reference video or the sign. Either of the first and second display processes may be performed as specified by the user.

[0012] In addition, in the second display step, the recorded video and the standby video, which is the video from a setting area other than the recording area, may be displayed on different displays.

[0013] Furthermore, the number of signs displayed in the display process may be variable, and identification information for each sign may be displayed in the display process. In that case, if the number of signs displayed is changed, it is preferable that the identification information set for each sign is maintained before and after the change in the number of signs displayed.

[0014] Furthermore, one embodiment of the present invention is a display method for displaying images captured by a shooting device, comprising: a setting step of setting a plurality of setting areas within a reference area which is the shooting area of ​​a reference image; a selection step of selecting a recording area which is the area of ​​the recorded image to be recorded from among the plurality of setting areas; a switching step of switching the recording area by re-selecting the recording area from among the plurality of setting areas after the selection step has been performed; a determination step of determining the priority of the plurality of standby areas when setting areas other than the recording area are set as standby areas; and a display step of displaying a plurality of standby images which are images of the plurality of standby areas, wherein when the number of plurality of standby areas set within the reference area reaches a set value related to the number of standby images to be displayed, the display step displays the standby image of the standby area selected from among the plurality of standby areas based on priority, or displays the plurality of standby images in a size according to priority.

[0015] Furthermore, in the decision-making process, subjects may be detected in multiple standby video feeds, and the priority of each of the multiple standby areas may be determined according to the information about the detected subjects.

[0016] Furthermore, in the decision-making process, the history of a standby area being selected as a recording area in the past may be identified, and the priority of each of the multiple standby areas may be determined based on the information regarding the identified history.

[0017] Furthermore, one embodiment of the present invention is a display method for displaying images captured by a camera, comprising: a setting step of setting a plurality of setting areas within a reference area which is the shooting area of ​​a reference image; a selection step of selecting a recording area which is the area of ​​the recorded image to be recorded from among the plurality of setting areas; a switching step of switching the recording area by re-selecting the recording area from among the plurality of setting areas after the selection step has been performed; a display step of displaying a plurality of setting images which are images of the plurality of setting areas; and a control step of executing control processing related to changing the reference area at least after the reference area has been determined.

[0018] Furthermore, the control process may include at least one of the following: a process to suppress changes in the reference area, a process to encourage changes in the reference area, and a process to notify the user of proposed information regarding changes in the reference area. Furthermore, if the imaging device is equipped with an optical device for zooming, the control process may include control processing related to zooming by the optical device.

[0019] Alternatively, the reference area may be determined when the user's recording start command is received, and the control process may involve processing related to changes in the reference area after the recording start command has been received.

[0020] Furthermore, one embodiment of the present invention is a display method for displaying images captured by a camera, comprising: a setting step of setting a plurality of setting areas within a reference area which is the shooting area of ​​a reference image; a selection step of selecting a recording area which is the area of ​​the recorded image to be recorded from among the plurality of setting areas; a switching step of switching the recording area by re-selecting the recording area from among the plurality of setting areas after the selection step has been performed; and a display step of displaying a setting image which is the image of the setting area, wherein at least one of the number of setting images displayed in the display step and the display size of the setting images is determined according to at least one of the resolution and aspect ratio of the recorded image.

[0021] Further, the number of setting videos displayed in the display process may be determined according to at least one item adjusted according to the recording format of the recorded video.

Brief Description of the Drawings

[0022] [Figure 1] It is a perspective view showing an example of the appearance of a photographing apparatus according to an embodiment of the present invention. [Figure 2] It is a rear view showing the rear side of a photographing apparatus according to an embodiment of the present invention. [Figure 3] It is a block diagram showing the configuration of a photographing apparatus according to an embodiment of the present invention. [Figure 4] It is an explanatory diagram of electronic shake correction. [Figure 5] It is a diagram showing the relationship between a reference area and a setting area. [Figure 6] It is a diagram showing the relationship between a reference video and a recorded video. [Figure 7] It is a diagram showing the recorded videos before and after switching among the moving images showing the recorded video. [Figure 8] It is a diagram showing a reference video when a new subject enters the shooting area. [Figure 9] It is a diagram showing how a moving area follows a moving subject. [Figure 10] It is a diagram showing the flow of the video display flow. [Figure 11] It is a diagram showing the transition of the UI screen in the setting process. [Figure 12] It is a diagram showing how to set a setting area when the previous use pattern is selected. [Figure 13] It is a diagram showing a selection screen for preset patterns. [Figure 14] It is an explanatory diagram of a user operation regarding the position of the setting area. [Figure 15] It is a diagram showing a setting area set based on an AF position specifying pattern. [Figure 16] It is a diagram showing a feature portion in a reference video. [Figure 17]This diagram shows how the setting area has been defined at the feature location. [Figure 18] This figure shows an example of the display screen when the display process is performed. [Figure 19] This figure shows an example of the display screen when the second display process is performed. [Figure 20] This figure shows a modified example of the display screen when the second display process is performed. [Figure 21] This figure (Part 1) shows an example screen when the number of standby areas reaches the maximum number of standby videos that can be displayed. [Figure 22] This diagram shows the flow of the display process when the number of standby areas reaches the maximum number of standby images that can be displayed. [Figure 23] This figure (part 2) shows an example screen when the number of standby areas reaches the maximum number of standby videos that can be displayed. [Figure 24] This diagram shows a modified version of the screen when the number of standby areas reaches the maximum number of standby images that can be displayed. [Figure 25] This diagram shows the processing flow during the recording process. [Figure 26] This figure shows an example of the screen before the change process was implemented. [Figure 27] This figure shows an example of the screen when the number of displayed setting videos increases due to the modification process. [Figure 28] This figure shows an example of the screen when the movement area overlaps with other setting areas. [Figure 29] This diagram shows an example screen where overlapping movement areas and other settings areas are merged into a single settings area. [Figure 30] This figure shows an example of the screen when the display of the settings video in a certain settings area is stopped. [Figure 31] This figure shows an example screen displaying suggested information prompting a change in the reference area. [Modes for carrying out the invention]

[0023] A preferred embodiment of the present invention (hereinafter referred to as "this embodiment") will be described in detail with reference to the accompanying drawings. This embodiment relates to a method for displaying images using the imaging device 10 shown in Figures 1 to 3. However, the embodiments described below are merely examples given to facilitate understanding of the present invention and do not limit it. That is, the present invention can be modified or improved from the embodiments described below without departing from its spirit. Furthermore, the present invention includes equivalents thereof.

[0024] [Basic configuration of the imaging device] The imaging device 10 is, for example, a digital camera that captures images. Here, "video" refers to live video (live view image), that is, video captured in real time, unless otherwise specified. Furthermore, the video in this embodiment is primarily a moving image. In other words, the shooting device 10 captures a moving image at a predetermined frame rate.

[0025] The imaging device 10 is, for example, a lens-interchangeable model, and comprises an imaging device body 12 and an imaging lens 14, as shown in Figures 1 and 2. The imaging lens 14 is interchangeably mounted to the mount 13 of the imaging device body 12. However, it is not limited to this, and the imaging device 10 may also be a lens-integrated model.

[0026] (Photography lens) As shown in Figure 3, the photographic lens 14 includes an optical unit 17, an aperture 20, a zoom drive unit 21, a focus drive unit 22, and an aperture drive unit 23. The optical unit 17 includes an optical instrument 18 for zooming (e.g., a zoom lens). By moving the optical instrument 18 in the direction of the optical axis L1, the shooting area of ​​the imaging device 10 expands or contracts, changing the zoom. In other words, the imaging device 10 is equipped with an optical zoom function using the optical instrument 18. Furthermore, the imaging device 10 of this embodiment is equipped with an optical zoom function as well as an electronic zoom (digital zoom) function that enlarges the subject in the image through electronic processing.

[0027] The optical unit 17 also includes an optical device 19 for focusing (for example, a focusing lens). By moving the optical device 19 in the direction of the optical axis L1, the focus position of the photographic lens 14 changes, and the focus is adjusted. Furthermore, the shooting device 10 of this embodiment is equipped with an autofocus (AF) function. That is, when the user performs a predetermined operation (for example, an instruction operation to start video recording) during video recording, the focus drive unit 22 operates and moves the optical device 19. As a result, the focus position is automatically adjusted so that it is in focus on a predetermined part of the image.

[0028] Furthermore, the optical unit 17 includes a wide-angle lens, an ultra-wide-angle lens, a 360-degree lens, or an anamorphic lens. As a result, the imaging device 10 can capture images with a wide field of view in the lateral direction (width direction) of the imaging device 10. The field of view of the imaging device 10 changes as optical zoom is performed, etc. The maximum field of view (total field of view) of the imaging device 10 is determined according to the design specifications of the optical unit 17 and the image sensor 40 described later. Incidentally, the optical unit 17 provided in the imaging device 10 is not limited to one, and may be provided with multiple optical units 17 that have different angles of view from one another.

[0029] The aperture 20 has a variable aperture shape, which is adjusted by the aperture drive unit 23. By adjusting the aperture shape, the amount of aperture relative to the light incident on the photographic lens 14 changes, thereby adjusting the exposure amount.

[0030] (Main unit of the imaging device) As shown in Figures 1 and 2, the camera body 12 includes an operating section with an operating button 30. A release button 26, which is one of the operating sections, is located on the top surface of the camera body 12. When the release button 26 is pressed, recording begins of the video captured by the camera body 10, or of a video extracted from that video (more precisely, the recorded video described later).

[0031] Here, the shooting area of ​​the camera 10, that is, the range that falls within the field of view of the camera 10, is determined when the user's recording start command is received. The shooting area also changes depending on, for example, whether electronic image stabilization is turned on or off. When electronic image stabilization is off when the recording start command is received, the shooting area is generally the full-frame (entire field of view) shooting area. Furthermore, the user's operation to initiate video recording is not limited to pressing the release button 26; it may also involve touching a predetermined position on the rear display 28, which will be described later.

[0032] Hereafter, the image captured by the camera 10 at its current field of view will be referred to as the "reference image." The area of ​​the reference image will be referred to as the "reference area."

[0033] The rear of the imaging device body 12 is provided with a display as shown in Figure 2 (hereinafter also referred to as the rear display 28). The rear display 28 is composed of, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electroluminescence) display, an LED (Light Emitting Diode) display, or electronic paper.

[0034] The rear display 28 displays reference images and recorded images, which will be described later. Thus, the shooting device 10, which has a display, is equipped with an image display function and is used as an image display device. Furthermore, the rear display 28 can display information other than video, such as advice or suggestions regarding shooting (see Figure 18).

[0035] In this embodiment, the camera body 12 includes an electronic viewfinder (indicated as EVF in Figure 3) 29 as a sub-display, as shown in Figures 2 and 3. The rear display 28 and the electronic viewfinder 29 can display the same image, or they can display different images. Incidentally, the camera does not necessarily have to include an electronic viewfinder 29. Also, it is not necessary to display an image on the electronic viewfinder 29 and an image on the rear display 28.

[0036] Furthermore, a touch panel 36 for detecting user operations is provided inside the rear display 28 or on the exposed surface of the display 28. For example, when a user touches a predetermined position on the rear display 28, the touch panel 36 detects the touch position and outputs the detection result to the control unit 46, which will be described later.

[0037] As shown in Figure 3, the main unit of the imaging device 12 is further provided with a shutter 38, an image sensor 40, an analog signal processing circuit 44, a control unit 46, an internal memory 50, a card slot 52, and a buffer memory 56.

[0038] The image sensor 40 is an image sensor that receives light that has passed through the photographic lens 14, converts the received image into an electrical signal (video signal), and outputs it. As the image sensor 40, a solid-state image sensor such as a CCD (Charged Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor Image Sensor) can be used.

[0039] The control unit 46 controls each part of the shooting device 10 and performs various processes related to image capture, display, and recording. As shown in Figure 3, the control unit 46 includes a controller 47 and an image processing unit 48.

[0040] The control unit 46 has one or more processors, and the control unit 46 is configured through the cooperation of the processors and the control program. The processors may be composed of, for example, a CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), GPU (Graphics Processing Unit), or other IC (Integrated Circuit). Alternatively, the processor may be configured by combining these. Furthermore, the processor may be configured using a single IC (Integrated Circuit) chip, such as a SoC (System on Chip), to encompass the entire functions of the control unit 46, including the controller 47 and the video processing unit 48. The hardware configurations of each processor listed above may also be implemented using electrical circuits (Circuitry) that combine circuit elements such as semiconductor devices.

[0041] The controller 47 controls each part of the imaging device 10 based on user operation or according to a predetermined control program. For example, the controller 47 controls the image sensor 40 and the analog signal processing circuit 44 to capture images at a predetermined frame rate.

[0042] The controller 47 controls the drive units 21-23, the image sensor 40, and the analog signal processing circuit 44, etc., so that the shooting conditions are as desired. The shooting conditions include focus, exposure, and white balance. In other words, the controller 47 enables the autofocus (AF), auto exposure (AE), and auto white balance (AWB) functions of the shooting device 10.

[0043] Furthermore, when the controller 47 receives a recording start command from the user, it controls the video processing unit 48 to start recording video.

[0044] The video processing unit 48 converts the analog video signal sent from the analog signal processing circuit 44 into digital video data, and performs gamma correction, white balance correction, etc., on the digital video data. The processed digital video data is compressed in a compression format compliant with a predetermined standard.

[0045] The video processing unit 48 generates compressed digital image data at a predetermined frame rate during shooting and acquires video (more precisely, frame images) from that data. The video acquired at this time corresponds to the reference video shot in the reference area.

[0046] Furthermore, the video processing unit 48, under the control of the controller 47, extracts the recording image from the reference image. The recording image is an image with a field of view smaller than that of the reference image, and represents an image of a portion of the reference area. The recording image is generated at the same frame rate as the reference image and is displayed, for example, on the rear display 28 or the electronic viewfinder 29.

[0047] Furthermore, the recorded video is stored on a recording medium. As a result, a video file of the recorded video is created. At this time, the resolution (number of pixels) and aspect ratio of the recorded video are determined according to the recording format (video format) of the recorded video. Furthermore, there are no particular restrictions on the recording format; it can be freely determined.

[0048] Furthermore, while electronic image stabilization is enabled (on), the video processing unit 48 extracts an image slightly smaller in size from the uncorrected image captured in the full field of view, as shown in Figure 4. If the extracted image shifts due to vibration or other reasons, the video processing unit 48 performs electronic image stabilization and shifts the position of the extracted area in the uncorrected image (for example, shifting it from the position of the solid line to the position of the dashed line in Figure 4). Furthermore, while electronic image stabilization is enabled, the extracted footage will be used as the reference footage.

[0049] Hereafter, unless otherwise specified, the operations and processes of the controller 47 and the video processing unit 48 will be described as the operations and processes of the control unit 46.

[0050] The internal memory 50 built into the camera body 12 and the memory card 54 that can be inserted into the card slot 52 are recording media, on which, for example, recorded video is stored. The internal memory 50 and memory card 54 may be located in a separate device (external) from the main unit of the camera 12. In that case, the control unit 46 may record the video onto an external recording medium via wired or wireless connection. The buffer memory 56 functions as the work memory of the control unit 46.

[0051] [Regarding recording and switching of recorded video] The recording device 10 records the recorded video extracted from the reference video onto a recording medium. Furthermore, the recording device 10 can switch (change) the recorded video by changing the extraction area in the reference video. This function will be explained with reference to Figure 5. Note that, for the sake of clarity, the subjects in the video are not shown in Figure 5.

[0052] When the camera 10 starts shooting, a reference image is displayed on the display (for example, the rear display 28). Normally, the entire field of view image becomes the reference image, and the area enclosed by the outermost edge in Figure 5 becomes the reference area As. Also, while electronic image stabilization is turned on, the extracted image extracted from the entire field of view image becomes the reference image, and the area enclosed by the dashed frame in Figure 5 becomes the reference area At.

[0053] Before or during recording, if the user performs a predetermined operation, the control unit 46 sets a setting area Ar within the reference areas As and At. The setting area Ar is a part of the reference areas As and At, and for example, the subject in the reference image is reflected within the setting area Ar (see Figure 6).

[0054] Furthermore, the position and size of the setting area Ar can be adjusted automatically or based on user operation. By performing predetermined operations on the screen, the user can change the position and size of the setting area Ar in which a subject is displayed, for example, according to the position and size of the subject in the reference video.

[0055] In this embodiment, the aspect ratio of the setting area Ar is determined to a value corresponding to the recording format (video format) of the recorded video. Therefore, when changing the size of the setting area Ar, the aspect ratio of the setting area Ar remains constant while the size is changed. However, the aspect ratio of the setting area Ar may be determined independently of the recording format of the recorded video, in which case the size of the setting area Ar may be changed while changing the aspect ratio.

[0056] When the setting area Ar is set, a marker f will be displayed in the reference image shown on the display at that time, as shown in Figure 5. Marker f indicates the position of the set region Ar in the reference image and includes the boundary of the set region Ar. The boundary of the set region Ar is set to distinguish (identify) the set region Ar from its surroundings.

[0057] Examples of markers f that include boundaries include graphic objects such as a frame surrounding the setting area Ar, and a pointer indicating a representative position within the setting area Ar. Differences in brightness, chromaticity, and shading between the setting area Ar and its surroundings also qualify as markers f. In addition, anything that distinguishes the setting area Ar from its surroundings may be included in the category of markers f. In the following explanation, we will use a marker f consisting of a colored rectangular frame as an example.

[0058] Multiple setting regions Ar can be set in the reference image, as shown in Figure 5. In this case, a marker f is displayed for each setting region Ar. In addition, identification information (the number displayed in the upper right corner of marker f in Figure 5) is displayed for each marker f. Identification information is information that the control unit 46 assigns individually to each marker f according to predetermined assignment rules. By displaying the identification information together with the marker f, the user can distinguish each marker f. The identification information is not particularly limited and may be a number, letter, or symbol, or it may be the display color of each individual identification f. If a number is used as the identification information, it is desirable that it be sequential.

[0059] Once the settings for setting area Ar are complete, the user selects a recording area Ap from among the multiple setting areas Ar that will contain the recorded video. The control unit 46 selects the recording area Ap from the multiple setting areas Ar according to the user's selection operation. This determines the setting area Ar, i.e., the recording area Ap, from which the video within that area will be recorded.

[0060] Once the recording area Ap is determined, the control unit 46 displays, in different ways, the marker f indicating the position of the recording area Ap and the marker f indicating the positions of the other setting areas Ar, which are displayed in the reference video. For example, the display modes of the marker f include the color, brightness, density (degree of darkness), shape, thickness and type of the border line, and whether or not it flashes. In the case shown in Figure 5, the color and thickness of the indicator f that shows the position of the recording area Ap are different from the other indicators f.

[0061] After the recording area Ap is determined, if the user issues a recording start command, recording of the video in recording area Ap (hereinafter referred to as "recorded video") begins. At that point, the reference area (i.e., the orientation and field of view of the shooting device 10) is determined. It should be noted that the determination of the reference area means that the reference area is determined once at the time the recording start command is received. Furthermore, even after the reference area has been determined, the user can freely change the reference area. Incidentally, when the user changes the reference area, it is preferable to change the position of the set area according to the position of the subject, as will be described later.

[0062] As described above, by recording a portion of the reference video as a recorded video, it is possible to record, for example, a zoomed-in video of a specific part of the reference video that is of interest. Although the recorded video has a smaller field of view than the reference video, if the reference video is high quality (for example, if the pixel count is 10 million or more), the recorded video will be of sufficiently high quality. The number of pixels in the reference image is not particularly limited, but its lower limit is preferably 10 million or more, and more preferably 60 million or more. The upper limit of the number of pixels is preferably 1 billion or less, and more preferably 500 million or less. If the number of pixels is greater than the lower limit above, the visibility of the recorded image is ensured. If the number of pixels is less than the upper limit above, the amount of data in the reference image is reduced, and processing by the control unit 46 is accelerated.

[0063] Furthermore, in this embodiment, after selecting one setting area Ar from among multiple setting areas Ar to be determined as the recording area Ap, it is possible to switch the recording area Ap to another setting area Ar during video recording. As a result, it is possible to record the video from the recording area Ap before the switch and the video from the recording area Ap after the switch. This will be explained in detail with reference to Figure 6.

[0064] Suppose the shooting device 10 photographs multiple subjects (people) in a certain scene (location) and sets a setting area Ar for each subject as shown in Figure 6. In this case, the one setting area Ar selected by the user from the multiple setting areas Ar becomes the recording area Ap. In the case shown in Figure 6, four setting areas Ar are set, and the setting area Ar surrounded by the identifier f of identification information "1" is the recording area Ap.

[0065] Under the circumstances described above, when the user issues a recording start command, the video from the setting area Ar, which is currently recording area Ap, will be recorded. Then, if the user gives an instruction to change the recording area Ap to another setting area Ar (for example, setting area Ar surrounded by the indicator f of identification information "2") during video recording, the recording area Ap is re-selected and the recorded video switches. Thus, in this embodiment, the recorded video can be switched during video recording by re-selecting the recording area Ap. This makes it possible to capture zoomed-in video of each subject without having to set up a separate camera for each of the multiple subjects.

[0066] Furthermore, when the recorded video is switched, the control unit 46 records the recorded video before the switch and the recorded video after the switch, and combines these videos into a single video. As a result, as shown in Figure 7, a video file is obtained in which the subject in the video changes in accordance with the switching of the recorded video. However, this is not limited to the above; the recorded video before the switch and the recorded video after the switch may be recorded as separate file data without being combined. Furthermore, the reference video and the standby area described later may also be recorded as separate file data.

[0067] In this embodiment, as shown in Figures 5 and 6, a marker f indicating the position in the reference image is displayed for each of the multiple setting areas Ar, including the recording area Ap. This allows the user to understand the position of each setting area Ar during shooting. In other words, the display method of this embodiment improves convenience (ease of use) by making it easier to understand the position of the setting areas Ar. This effect is particularly effective in configurations where it is possible to switch between recording areas Ap among multiple setting areas Ar. That is, by displaying a marker f for each setting area Ar, it is easy to understand which part of the reference image the recorded video after switching corresponds to.

[0068] To elaborate on the setting area Ar, the number of setting areas Ar (setting count) can be changed automatically or based on user input, for example, when the number of subjects in the reference image changes. For example, as shown in Figure 8, if a new subject enters the reference image, a new setting area Ar can be added at the position of that subject. Conversely, if a subject in the reference image moves outside the reference image, the setting area Ar that occupied the image area of ​​that subject can be automatically deleted.

[0069] Furthermore, if a subject moves within a certain set area Ar (for example, the set area Ar enclosed by the marker f of identification information "5" in Figure 8), the set area Ar can be displaced to follow the movement of the subject. This makes it possible to track a moving subject in the reference image. Here, subject tracking can be achieved using known subject detection techniques based on video analysis. Furthermore, if the setting area Ar, which tracks a moving subject, is the recording area Ap, then video of the moving subject can be recorded.

[0070] Hereafter, the setting region Ar that tracks a moving subject will be referred to as the moving region Am. Note that the moving region Am is not limited to a region that moves in accordance with a moving subject, but may also be a region that moves at a constant speed or irregular speed in a predetermined direction.

[0071] [Video display flow] The processing flow (hereinafter referred to as the video display flow) executed by the control unit 46 of the imaging device 10 will be explained with reference to Figure 10.

[0072] When the user turns on the power to the camera 10, the video display flow starts. Before starting the video display flow, the user performs initial adjustments to the camera 10. During these initial adjustments, the user adjusts the resolution (number of pixels) and aspect ratio of the recorded video, for example, by specifying the recording method. Such adjustments are performed, for example, when the camera 10 is first started up, and these adjustment settings are retained thereafter. The above adjustments may be performed by the equipment manufacturer before the imaging device 10 is shipped. Furthermore, the user can readjust the resolution, aspect ratio, etc., after the imaging device 10 has been started, in order to review the settings once they have been adjusted.

[0073] In the video display flow, first, the control unit 46 starts the shooting process (S001). This shooting process is the process of capturing a reference image in the reference area at that time. Preferably, as soon as shooting starts, the real-time reference image is displayed as a live view image on the rear display 28 or the like. The shooting process continues until the user instructs to end shooting or the power of the shooting device 10 is turned off.

[0074] Subsequently, the control unit 46 performs a setting process (S002). The setting process is a process of automatically setting multiple setting areas Ar within the shooting area of ​​the reference image. In the setting process, the initial position and initial size of each setting area Ar are set, and thereafter, the position and size of each setting area Ar can be changed automatically based on user operation or according to predetermined rules.

[0075] After the setting process is completed, the control unit 46 performs a selection process (S003). The selection process is the process of selecting a recording area Ap from among the multiple setting areas Ar set in the setting process, based on the user's selection operation. When performing the selection process, the control unit 46 may display the respective images of the multiple setting areas Ar (hereinafter referred to as "setting images") on the display. In this case, the user can check the setting images for each of the multiple setting areas Ar.

[0076] Subsequently, when the user issues a recording start command (S004), the control unit 46 starts the recording process (S005). The recording process is the process of recording the video, which is the video of the selected recording area Ap, onto the recording medium. The video is recorded according to a predetermined recording format (video format). Furthermore, as mentioned above, the reference area is determined at the time of receiving the recording start instruction, which triggers the start of the recording process.

[0077] Furthermore, during the recording process, the control unit 46 performs a display process (S006). The display process involves displaying multiple setting images, including the recorded image, the reference image at that time, and the marker f in the reference image on the display. By viewing these images, the user can confirm the position of each setting area Ar in the reference image and understand which setting area Ar is the recording area Ap.

[0078] Furthermore, during the recording process, the control unit 46 performs a control process (S007). The control process is a process of executing control processing related to the change of the reference area determined at the time of receiving the recording start instruction. In this embodiment, the control processing includes processing to suppress changes in the reference area, for example, control processing related to zoom (optical zoom) by the optical device 19.

[0079] Furthermore, if the user instructs the recording area Ap to be re-selected during the recording process (S008), the control unit 46 performs a switching process (S009). The switching process is a process in which, after the selection process has been performed, the recording area Ap is re-selected from among multiple setting areas Ar in accordance with the user's re-selection instruction, and the recording area Ap is switched. If the switching process is performed, the video (recorded video) of the switched recording area will be recorded in the subsequent recording process.

[0080] The series of steps from the recording process (i.e., S005 to S009) are then repeated until the user instructs the end of video recording (S010). On the other hand, when the user instructs the end of video recording, the recording process ends at that point. If a recording termination instruction is given thereafter, or if the power to the recording device 10 is turned off (S011), the video display flow ends at that point.

[0081] [About the setup process] This section explains the setup process in the video display flow. In this embodiment, when the setting process is executed, the first UI (User Interface) screen 101 shown in Figure 11 is displayed on the rear display 28. The user selects either the "recording pattern" or the "automatic setting pattern" as the setting pattern for the setting area Ar, and presses one of the buttons Bt1 or Bt2 on the UI screen 101 that indicates the selected pattern.

[0082] In the setting process when a recording pattern is selected, the control unit 46 reads the position information of the setting area Ar that has been pre-stored in the shooting device 10. Then, the setting area Ar is automatically set to the position indicated by the read position information. At this time, it is preferable to highlight the image within the marker f in the reference image in order to make the marker f easier for the user to see. For example, the brightness of the image outside of the marker f (i.e., within the boundary of the setting area Ar) in the reference image may be reduced or grayed out (see Figure 18).

[0083] In this embodiment, when a recording pattern is selected, a second UI screen 102 is displayed on the rear display 28, as shown in Figure 11. The user selects one of several pre-prepared recording patterns and presses one of the buttons Bt3, Bt4, or Bt5 on the second UI screen 102 that indicates the selected pattern. For example, the recording patterns that can be selected are "previously used pattern," "preset pattern," and "user input pattern."

[0084] On the other hand, in the setting process when the automatic setting pattern is selected, the control unit 46 automatically sets the setting area Ar at a predetermined position in the reference image based on the reference image being captured. In this embodiment, when an automatic setting pattern is selected, a third UI screen 103 is displayed as shown in Figure 11. The user selects one of several pre-prepared automatic setting patterns and presses one of the buttons Bt6, Bt7, or Bt8 on the third UI screen 103 that indicates the selected pattern. Examples of selectable automatic setting patterns include "scene recognition pattern," "subject detection pattern," and "AF position identification pattern." In the case of an automatic setting pattern, it is preferable that at the start of the setting process, the markers f are displayed so that each setting area Ar does not overlap or superimpose on each other. This makes it easier to identify multiple markers f and facilitates the setting of the setting areas Ar during the setting process.

[0085] The following describes each of the multiple recording patterns and multiple automatic setting patterns.

[0086] (Previous usage pattern) If the previously used pattern is selected, the control unit 46 reads the position information of the setting area Ar set in the setting step of the previous (immediately preceding) video display flow. This position information indicates the position of each of the multiple setting areas Ar set in the setting step of the previous video display flow, and is stored, for example, in the internal memory 50 of the shooting device 10. Preferably, the stored position information is retained even after the power of the shooting device 10 is turned off.

[0087] Furthermore, the size information of the setting area Ar may be stored together with the position information of the setting area Ar. In that case, when the previously used pattern is selected, it is desirable that the position information and size information be read out together.

[0088] Then, as shown in Figure 12, the control unit 46 automatically sets the setting area Ar to the same position as indicated by the read position information, that is, the same position as the setting position in the previous video display flow. This previous usage pattern is effective when shooting video with the same composition as the previous video display flow (for example, when shooting video at the same angle in the same location).

[0089] In the above embodiment, the read position information indicates the position of the setting area Ar set in the previous (immediately preceding) video display flow, but it is not limited to this. For example, the position information of the setting area Ar set in several setting processes performed in the past may be stored. In that case, the current setting area Ar should be set to the position of the setting area Ar set in the setting process specified by the user from among the stored position information.

[0090] (Preset patterns) If a preset pattern is selected, the control unit 46 reads the position information of the setting area Ar, which is arranged in a predetermined fixed arrangement pattern. The fixed arrangement patterns are pre-registered in the imaging device 10, or are newly registered during program updates after purchase.

[0091] When position information related to a standard arrangement pattern (preset pattern) is read, the fourth UI screen 104 shown in Figure 13 is displayed on the rear display 28. Multiple types of patterns (patterns A to D in Figure 13) are displayed for selection on the fourth UI screen 104, and the user selects one of the preset patterns. The control unit 46 automatically sets the setting area Ar at a position corresponding to the preset pattern selected by the user. Such preset patterns are useful when shooting video with a predetermined composition (for example, when shooting video at a specific location) and there is a pattern that corresponds to that composition. Furthermore, the number of preset patterns, as well as the number and placement of setting areas Ar within each preset pattern, are not particularly limited and can be determined arbitrarily.

[0092] (User input pattern) When a user input pattern is selected, the control unit 46 reads the position information of the setting area Ar determined based on the user's operation. The user's operation here is an input operation to register the position of the setting area Ar, and is performed in a preliminary stage of the setting process (for example, the initial registration stage for the pattern). The above input operation is performed, for example, through the input screen 105 for pattern registration shown in Figure 14. The user touches and drags the inner area of ​​the area setting indicator fx displayed on the input screen 105. As a result, the display position of the indicator fx after dragging is registered as the position of the setting area Ar.

[0093] The control unit 46 automatically sets the setting area Ar to the position indicated by the read position information, that is, the setting position determined based on the user's input operation. Such user input patterns are useful when a pattern is prepared based on user input in situations where the aforementioned preset patterns cannot be used.

[0094] Furthermore, when the position of the setting area Ar is registered, the size of that setting area Ar may also be registered. In other words, information indicating the position and size of the setting area Ar determined based on the user's input operation may be stored. In that case, each of the multiple setting areas Ar will be set to the position and size registered as a user input pattern.

[0095] (Scene recognition pattern) In the setting process when a scene recognition pattern is selected, the control unit 46 recognizes the shooting scene of the reference video based on the reference video. The shooting scene of the reference video may be, for example, the type of scenery, event, subject, scene (e.g., whether it is a daytime scene or a night scene), and circumstances (e.g., weather).

[0096] Furthermore, known scene recognition technologies can be used as methods for recognizing shooting scenes based on reference footage. For example, one method is to determine whether the brightness or color of a predetermined area of ​​footage falls within a preset range. Alternatively, machine learning may be performed using the correct labels indicating the shooting scenes of the footage and the footage data as a training dataset, and the scene recognition model obtained as a result of the training may be used to recognize the shooting scenes of the reference footage.

[0097] (Subject detection pattern) In the setting process when a subject detection pattern is selected, the control unit 46 detects a subject in the reference image and automatically sets the setting area Ar at the position of the detected subject. A subject in the reference image is any subject other than the scenery in the reference image that can be detected by the control unit 46, and includes, for example, people, animals, and vehicles. When detecting a subject, known template matching techniques may be applied. For example, an image of the subject may be stored in advance as a template image, and the template image may be compared with each part of the reference video to detect the subject if the video is similar to the template image. Alternatively, a model that recognizes subjects in a video may be constructed using machine learning, and this model may be used to detect subjects in the reference video. As the algorithm for the subject recognition model, known algorithms such as YOLO (You Only Look Once) or R-CNN (Region with Convolutional Neural Network) can be used.

[0098] In the subject detection pattern, the control unit 46 can set one setting region Ar for each detected subject, as shown in Figure 6. At this time, the entire corresponding subject may be contained within each setting region Ar, or only a part of the corresponding subject (for example, the upper body including the face) may be contained within it.

[0099] According to the subject detection pattern described above, the setting area Ar can be automatically set for a subject in the reference video without requiring any effort from the user. In this embodiment, since the video of the recording area Ap selected from among multiple setting areas Ar is recorded, the subject detection pattern is effective when recording the video of a subject in the reference video.

[0100] (AF position identification pattern) In the setting process when an AF position identification pattern is selected, the control unit 46 identifies a position in the reference image where the features for determining the focus of the shooting device 10 satisfy the criteria, and automatically sets the setting area Ar at the identified position. The features for determining the focus are, for example, features for determining the focus position in the reference image. To give a specific example, if the autofocus is a phase-detection AF (Auto Focus) method, the phase difference corresponds to the above features. Also, if the autofocus is a contrast AF method, the contrast between multiple pixels corresponds to the above features. The AF method may be an AF method using directional light, or an AF method using DFD (Depth from defocus).

[0101] Furthermore, a position that satisfies the criteria is a position where the above-mentioned characteristics (e.g., phase difference or contrast) become characteristics of focus. To put it simply, the control unit 46 identifies a position in the reference image that satisfies the criteria, which is the position where focus is achieved by the AF function. Then, as shown in Figure 15, the setting area Ar is automatically set for the identified position (the position where focus is achieved, which is the hatched position in Figure 15). At this time, it is preferable that the size of the setting area Ar is also automatically set according to, for example, the area of ​​the in-focus part in the reference image.

[0102] The portion of the image that is in focus due to the AF function is generally the part that is recorded. Considering this, by setting the setting area Ar to the in-focus position, it becomes easy to record the in-focus image, thus improving convenience for the user.

[0103] In the AF position identification pattern, when the control unit 46 sets a setting area Ar at the position where focus is achieved, it automatically sets another setting area Ar accordingly. For example, as shown in Figure 15, another setting area Ar is set at a symmetrical position to the position where focus is achieved (for example, a point-symmetric position when viewed from the center of the reference image). This allows multiple setting areas Ar to be automatically set within the reference area when the AF position identification pattern is selected. However, the positions where another setting area Ar is set are not limited to the above-mentioned positions.

[0104] Furthermore, if the entire reference image is in focus in the AF position identification pattern, the entire reference area may be set as a single setting area Ar. Furthermore, if no subject is detected in the reference image and a suitable location for setting area Ar cannot be found within the reference image, setting area Ar may be set near the center of the reference image.

[0105] In this embodiment, the focus position within the reference image is given as an example of the shooting conditions, and the setting area Ar is set at the position in the reference image where the features for determining the focus position satisfy the criteria (i.e., the position where the image is in focus). Here, the shooting conditions do not have to be the focus position, and for example, exposure amount and white balance can also be mentioned. However, shutter speed and ISO sensitivity are excluded from the shooting conditions as referred to here.

[0106] Here, taking exposure, one of the shooting conditions, as an example, a setting area Ar can be set in the reference image at a position where the characteristics for determining the exposure (e.g., pixel value or brightness) meet the criteria. For example, it is good practice to set a setting area Ar in the reference image at a position where the exposure has been adjusted to an appropriate value by the AE (Automatic Exposure) function. Alternatively, a setting area Ar may be set in the reference image at positions where the exposure is relatively low and at positions where the exposure is relatively high.

[0107] Furthermore, taking white balance, one of the shooting conditions, as an example, a setting area Ar can be set in the reference image at a position where the characteristics for determining white balance (e.g., chromaticity) meet the criteria. For example, it is good practice to set a setting area Ar in the reference image at a position where the white balance has been adjusted to an appropriate value by the AWB (Auto White Balance) function. To give a specific example, if there are areas in the reference image that are illuminated by light from multiple light sources such as lighting and the sun, a setting area Ar may be set for each illuminated area of ​​each light source.

[0108] The user selects any pattern from the multiple arrangement patterns described above. This ensures that the setting area Ar is configured in the setup process to reflect the user's intentions or preferences. Furthermore, since the setting area Ar is automatically configured during the setup process, it requires no user intervention (configuration work). In other words, the display method of this embodiment improves convenience (ease of use) by reducing user effort and setting the setting area Ar within the reference area to reflect the user's intentions, etc. Note that the arrangement pattern may be a combination of multiple arrangement patterns shown in Figure 11. For example, multiple setting areas Ar may be configured by simultaneously employing a scene recognition pattern and an AF position identification pattern.

[0109] The patterns described above are merely examples, and other arrangement patterns may be added, or conversely, some of the patterns described above may be omitted.

[0110] Furthermore, in the setting process of this embodiment, regardless of the type of pattern selected, if a characteristic area described later exists in the reference image, a candidate setting area Ac is automatically set at the position of the characteristic area within the reference area. Then, when the user performs a predetermined operation on the set candidate Ac, the setting area Ar is set at the position of the set candidate Ac. The following describes the procedure for setting up candidate Ac during the configuration process.

[0111] (Regarding setting up candidates) In the setting process, the control unit 46 determines whether or not a feature area exists in the reference image based on the characteristics of each pixel (e.g., pixel value) of the reference image. A feature area is a location in the reference image where a visual change occurs (excluding those detected as subjects), such as a location in the background image where there is movement, or a location where the color and brightness change. Specific examples include locations with flowing water like a waterfall, locations where lights like illuminations turn on or off, and locations where displayed characters and images like an electronic billboard switch. In the reference video shown in Figure 16, for example, the illuminated decorations appearing in the upper left part of the figure correspond to the characteristic feature.

[0112] If a characteristic area exists in the reference image, the control unit 46 performs a first setting step. The first setting step is a step in which candidate Ac of the setting area is automatically set at the location of the characteristic area in the reference image.

[0113] If the first setting step is performed, the subsequent display step will display a marker (hereinafter referred to as candidate marker K) indicating the position of candidate Ac within the reference image, as shown in Figure 16. Candidate marker K is displayed in a different manner from marker f, which indicates the position of setting area Ar. For example, candidate marker K in Figure 16 is a roughly L-shaped graphic object representing the rectangle of the rectangular area surrounding the feature area. The type and shape of candidate marker K are not particularly limited; for example, it may be a pointer in the shape of an arrow or a finger, or a graphic object consisting of symbols such as a circle or a star. Alternatively, the difference in density (difference in brightness) between candidate Ac and its surroundings in the reference image may be used as candidate marker K.

[0114] If the user performs an operation such as tapping candidate marker K during the display process, this triggers the execution of the second setting process. The second setting process is a process in which the control unit 46 sets a setting area Ar at the setting position of candidate Ac where the user's operation on candidate Ac (e.g., a tap operation) was performed. As a result of the execution of the second setting process, a new setting area Ar is set in the area of ​​the feature area of ​​the video, as shown in Figure 17.

[0115] As described above, in this embodiment, candidate Ac for the setting area can be automatically set for feature areas in the reference video that are not recognized as subjects but where visual changes occur. Furthermore, by displaying candidate markers K indicating the position of the set candidate Ac in the reference video, the user can easily confirm the candidate Ac. In addition, by performing a predetermined operation on candidate Ac, the setting area Ar is set at the position of candidate Ac. This allows the user to set the setting area Ar for feature areas of interest in the reference video. Moreover, by selecting the set setting area Ar as the recording area, it becomes possible to record the video of the feature area.

[0116] [Regarding the display process] The display process within the video display flow is explained in detail below. When the setting area Ar is set during the setting process, the control unit 46 starts the display process. Thereafter, the control unit 46 continues the display process until the recording process is completed (more precisely, until the shooting is completed).

[0117] The display process involves displaying the recorded video, the reference video, and markers f indicating the position of each of the multiple set regions Ar within the reference video on a display. In the display process, markers f are displayed for each set region Ar within the reference video, as shown in Figure 18. This allows the user to accurately grasp the position of each set region Ar in the reference video.

[0118] Furthermore, in the display process, as shown in Figure 18, the video of each of the setting areas Ar other than the recording area Ap (hereinafter referred to as standby video) can be displayed. Furthermore, in the display process when the aforementioned feature location is present in the reference video, a candidate marker K indicating the location of the feature location is displayed in the reference video together with marker f.

[0119] To illustrate an example of the display process, recorded video, reference video, and standby video can be simultaneously displayed on the screen of the rear display 28 (hereinafter referred to as the display screen). For example, as shown in Figure 18, the display screen is divided into a main screen G1, a sub-screen G2, and an information display screen G3. In this case, the standby video can be displayed on the sub-screen G2.

[0120] The main screen G1 occupies a relatively large area of ​​the display screen (for example, more than half of the display screen) and displays the reference video. Preferably, the aspect ratio of the main screen G1 is set to the same value as the aspect ratio of the reference video. Sub-screen G2 is smaller than the main screen G1 and is arranged in a row along the vertical or horizontal direction of the display screen. Each sub-screen G2 displays either recorded video or a single standby video. Preferably, the aspect ratio of each sub-screen G2 is set to the same value as the aspect ratio of the recorded video and the standby video, respectively. Information display screen G3 is a screen that displays text information other than video, and for example, as shown in Figure 18, it displays advice regarding video recording.

[0121] Furthermore, the layout of the display screen (for example, the size and placement of the main screen G1 and sub-screen G2) is not limited to the configuration shown in Figure 18, and can be freely designed according to the specifications of the equipment or the user's preferences.

[0122] Furthermore, the main screen G1 displays markers f for each of the multiple setting areas Ar set during the setting process, along with the reference image. The number of markers f displayed on the main screen G1 during the display process is variable, and the maximum number of markers f that can be displayed (maximum display limit) can be displayed simultaneously. Furthermore, to make the marker f easier for the user to see, it is preferable to highlight the image within marker f in the reference image. For example, as shown in Figure 18, the brightness of the image outside of marker f (i.e., within the boundary of the set area Ar) may be reduced or grayed out in the reference image.

[0123] In the display process, an indicator f showing the location of the recording area Ap and an indicator f showing the location of the other setting area Ar (hereinafter also referred to as the standby area) are displayed in different manners. At this time, it is preferable that the indicator f showing the location of the recording area Ap is highlighted. For example, the border color may be made a relatively conspicuous color such as red or fluorescent, the border may be made thicker, or it may be displayed as flashing. Alternatively, the video within the recording area Ap (i.e., the recorded video) itself may be highlighted.

[0124] Furthermore, if there is a moving subject in the reference image, the movement region Am set for that subject moves in accordance with the subject. In the display process, the marker f indicating the position of the movement region Am is displayed in a different manner from the marker f indicating the position of the setting region Ar other than the movement region Am, as shown in Figure 9. This makes it easy to distinguish between the movement region Am and the other setting region Ar, and to easily find the position of the movement region Am in the reference image.

[0125] Furthermore, when a subject within the moving area Am moves in the depth direction (i.e., in the direction of approaching or moving away from the shooting device 10), the size of the subject in the reference image changes. Consequently, the ratio of the size of the subject to the size of the moving area Am (hereinafter referred to as the subject ratio) changes. In this case, the control unit 46 uses the electronic zoom function to change the size of the moving area Am in accordance with the change in the subject ratio, and adjusts it to keep the subject ratio constant.

[0126] Furthermore, as the subject moves, the movement area Am may move to the vicinity of the edge of the reference area (i.e., the outer edge of the field of view) or extend beyond the reference area. In such cases, it is preferable for the control unit 46 to display a warning message on the information display screen G3 or to output a warning sound to inform the user of the above situation.

[0127] Furthermore, when the reference image (i.e., the extracted image extracted from the full-angle image) changes due to electronic image stabilization, the relative position of each setting area Ar with respect to the reference area changes (see Figure 4). In this case, the control unit 46 calculates the amount of shift in the reference image due to electronic image stabilization and adjusts the position of each setting area Ar based on the calculated amount of shift. This ensures that the position of each setting area Ar with respect to the reference area is maintained before and after the implementation of electronic image stabilization. Furthermore, it would be preferable if the function to adjust the position of the setting area Ar in conjunction with electronic image stabilization could be freely switched on or off according to the user's operation.

[0128] Furthermore, in the display process, as shown in Figure 18, the identification information of each marker f is displayed for each marker f. The identification information of each marker f is fixed during the display process, and remains unchanged even if the number of marker f displayed changes during the display process. For example, in Figure 18, if the display of marker f with identification number "3" is stopped, the identification numbers of the other marker f (i.e., "1", "2", and "4") remain unchanged. By fixing the identification number of each marker f during the display process in this way, it is easy to understand which setting area Ar is located where, even if the number of marker f displayed changes.

[0129] Multiple sub-screens G2 display multiple setting videos, including recorded video and standby video. Specifically, each of the multiple sub-screens G2 displays one corresponding setting video. If the aspect ratio of each sub-screen G2 is the same as the aspect ratio of the corresponding setting video, the setting video can be displayed neatly on the screen.

[0130] Furthermore, among the multiple setting videos displayed, it is preferable that the recorded video be highlighted. For example, it is preferable that the color of the screen frame, the thickness or type of the frame line, or whether or not it is illuminated differs between the sub-screen G2 displaying the recorded video and the sub-screen G2 displaying the other setting videos. This allows users viewing multiple setting videos on the display to easily distinguish which video is the recorded video.

[0131] Furthermore, as shown in Figure 18, it is advisable to display information indicating that the video is of the moving area Am (for example, the text "Moving") on sub-screen G2 (the second sub-screen from the top in Figure 18), which displays the video of the moving area Am.

[0132] In this embodiment, multiple display process execution modes are provided, and the method of displaying the video in the display process changes depending on the mode. In other words, the display process is performed according to the mode specified by the user. For example, if the user selects the first mode, the first display process is performed. In the first display process, as shown in Figure 18, the reference video and indicator f for each set area Ar are displayed on the main screen G1, and the recorded video and standby video are displayed on the sub-screen G2. On the other hand, if the user selects the second mode, the second display process is performed. In the second display process, as shown in Figure 19, the recorded video and standby video are displayed, but the reference video and indicator f for each set area Ar are not displayed.

[0133] In the first display step, the recorded video, standby video, reference video, and the indicator f for each set area Ar do not necessarily have to be displayed on the same display (for example, the rear display 28). In other words, as shown in Figure 20, the video displayed on the main screen G1 and the video displayed on the sub-screen G2 may be displayed on different displays. For example, the video to be displayed on the main screen G1 may be displayed on the rear display 28, and the video to be displayed on the sub-screen G2 may be displayed on the electronic viewfinder 29 or an external monitor (not shown) connected to the shooting device 10.

[0134] In this embodiment, the display process includes a first display process and a second display process, and the user specifies which of these two processes to perform. This allows the displayed image to be switched according to the user's needs. For example, if the user wants to check the reference image and marker f, the first display process should be performed to display the reference image and marker f. On the other hand, if the user wants to check only the recorded image and standby image (i.e., the set image), the second display process should be performed to display only the recorded image and standby image.

[0135] Furthermore, it is preferable that the execution mode of the display process, that is, which of the first and second display processes is performed, can be freely switched during the execution of the display process in response to a switching operation by the user. For example, it is preferable that the user can switch which display process to adopt via the touch panel 36 or operation button 30 shown in Figure 2.

[0136] In the second display step, the recorded video and other setting videos may be displayed on the same display. For example, as shown in Figure 19, the recorded video may be displayed on the main screen G1 of the display screen, and the standby video may be displayed on the sub-screen G2. Alternatively, in the second display step, the recorded video and other setting videos may be displayed on different displays. For example, as shown in Figure 20, the recorded video may be displayed on the rear display 28, and the other setting videos may be displayed on the electronic viewfinder 29 or an external monitor (not shown) connected to the shooting device 10.

[0137] The information display screen G3 shows, for example, the proposed information described later. Furthermore, the information display screen G3 can display shooting advice and warning messages to the user. For example, if adjustments to exposure or ISO sensitivity are necessary during the recording process, this may be displayed. Furthermore, the information display screen G3 can display information other than that mentioned above. For example, it may display the currently used shooting mode, current values ​​for shooting conditions such as resolution and ISO sensitivity, and the on / off status of various functions such as electronic zoom.

[0138] (Regarding the display of pre-set video based on priority) In the display process, the number of sub-screens G2 is determined according to at least one of the following items (hereinafter simply referred to as "at least one item"): the resolution and aspect ratio of the recorded video. In other words, the number of setting videos displayed on sub-screen G2 is determined according to at least one of these items. Similarly, the size of each sub-screen G2 is determined according to at least one of the settings. In other words, the display size of the setting video will be the size corresponding to at least one of the settings.

[0139] Here, at least one of the items is adjusted according to the recording format (video format) of the recorded video. Therefore, the number of images to display (more precisely, the maximum number of images to display) and the display size of the set video in the display process will be the number and size according to the recording format of the recorded video.

[0140] In this embodiment, the number and size of the set video to be displayed are determined according to at least one of the items, so that each of the multiple set videos, including the recorded video, can be displayed well. For example, each set video can be displayed at an appropriate resolution and fit nicely on the sub-screen G2.

[0141] In this embodiment, the aspect ratio of the recorded video is determined according to the recording format of the recorded video. However, the aspect ratio of the recorded video may be set arbitrarily without depending on the recording format. In that case, the number and size of sub-screens G2 will be changed according to the adjusted aspect ratio as the aspect ratio of the recorded video is adjusted.

[0142] Furthermore, in this embodiment, both the number and size of the setting video displayed in the display step (i.e., the number and size of sub-screens G2) are determined according to at least one of the items. However, this is not limited to this, and either the number or size of the setting video may be determined according to at least one of the items.

[0143] Furthermore, the number of images displayed in the display process does not necessarily have to be determined by at least one of the items. For example, the user may set or change the number of images displayed.

[0144] In this embodiment, the number of set images to be displayed in the display step is determined according to the resolution and aspect ratio of the recorded image, as described above. Here, in the case of the first display step, one of the set images to be displayed is the recorded image, and the rest are standby images. Therefore, the set value for the number of standby images to be displayed, i.e., the maximum number to be displayed, is determined according to the resolution and aspect ratio of the recorded image, and specifically, it is one less than the number of sub-screens G2. In the case shown in Figure 21, the maximum number of standby images displayed is three.

[0145] On the other hand, the number of setting areas Ar in the reference area changes depending on the subject in the reference video, and consequently, the number of setting areas Ar other than the recording area Ap, i.e., the number of standby areas, changes. In the case shown in Figure 21, the number of standby areas is 6.

[0146] When the number of standby areas reaches (or more precisely exceeds) the maximum number of standby images that can be displayed, the control unit 46 performs the display process according to the flow shown in Figure 22. To explain the flow of the display process shown in Figure 22, the control unit 46 performs the determination process (S021) when the number of standby areas reaches the maximum number of standby images that can be displayed.

[0147] In the decision-making process, the control unit 46 determines the priority for each of the multiple standby areas. To explain the procedure, the control unit 46 first detects a subject in each of the multiple standby video (S101). Next, the control unit 46 acquires information about the detected subject in the standby video (S102). At this time, the control unit 46 may acquire actual information for the recording area, which will be described later, instead of information about the subject in the standby video.

[0148] Information regarding the subject may include, for example, the duration or number of times the subject appears in the reference image during the shooting period, as well as the size (display size) of the subject in the reference image. This information can be obtained by the control unit 46 analyzing the reference image and measuring or counting it during the shooting period. Furthermore, if the subject is a person, the person's face may be recognized using techniques such as pattern matching, the importance of the recognized person may be determined, and the results of this determination may be used as information about the subject. The importance of a person can be determined, for example, by determining the attributes of the recognized person (e.g., family, relative, acquaintance, or stranger) and deriving the result from that determination. Also, if there are multiple detected subjects, priority may be determined by the category of the subject (e.g., person, animal, object, or landscape). Furthermore, priority may be determined by prioritizing subjects that were detected earlier. Of course, other information about the subject may also be considered besides that mentioned above. Ideally, users should be free to choose what information to use as subject information during the decision-making process.

[0149] Subsequently, in the determination step, the control unit 46 determines the priority of each of the multiple standby areas according to the information about the detected subject (S103).

[0150] After the decision process is completed, the control unit 46 selects N standby areas from among the multiple standby areas based on priority (S022). Here, the number N of standby areas to be selected is the maximum number of standby video displays, which is determined according to the number of sub-screens G2 (for example, a number that is 1 less than the number of sub-screens G2). In step S022, the control unit 46 selects N standby areas in order from the highest priority.

[0151] The control unit 46 then displays the images of the selected N standby areas (standby images) on the sub-screen G2 (S023). Here, if the sub-screens G2 are arranged vertically on the display screen as shown in Figure 21, it is desirable to display the standby images of the standby areas with higher priority on the upper sub-screens G2.

[0152] As described above, in this embodiment, when the number of standby areas reaches (exceeds) the maximum number of standby videos that can be displayed, N standby videos selected according to priority are displayed. This makes it possible to appropriately display high-priority standby videos while taking into account screen display constraints.

[0153] Furthermore, the method for determining priority is not limited to determining it based on information about the subject in the standby video. For example, priority may be determined based on past performance information of the recording area. To explain in more detail, for each of the multiple standby areas, the number of times it has been selected as a recording area from the start of shooting to the present time, the amount of time it was a recording area, or the elapsed time since it was last selected as a recording area are identified. This information corresponds to information about its past performance as a recording area. Furthermore, the priority of each of the multiple standby areas may be determined based on the performance information described above. For example, if the performance information is the number of times or the duration for which an area was selected as a recording area, a higher number of times or durations should be assigned a higher priority.

[0154] Furthermore, while the display process shown in Figure 22 involves selecting N standby areas based on priority and displaying only the video from the selected standby areas, it is not limited to this. For example, the top N standby areas and the lower-priority standby areas could be selected in order of priority. In that case, the standby video from the top N standby areas could be displayed first, and the standby video from the lower-priority standby areas could be displayed by scrolling or switching the display screen, as shown in Figure 23.

[0155] Furthermore, each of the multiple standby videos may be displayed in a size corresponding to its priority. For example, as shown in Figure 24, the standby video for a high-priority standby area may be displayed on a normal-sized sub-screen G2, while the standby video for a low-priority standby area may be displayed on a reduced-size sub-screen G2x. In this way, each of the multiple standby videos can be displayed, and the high-priority standby area can be displayed more clearly.

[0156] [Regarding the control process] The control process in the video display flow is described below. The control process is the process of executing control processing to suppress changes in the reference area, and is performed at least after the reference area has been determined. In other words, the control processing is the process of suppressing changes in the reference area after the user's recording start instruction has been received.

[0157] During the control process, the control unit 46 locks the zoom drive unit 21 to stop the movement of the zoom optical device 18. As a result, the optical zoom function is restricted during the recording process, the reference area is fixed, and the angle of view at that point is maintained. By executing the control process described above, it is possible to prevent the relative position of each setting area Ar with respect to the reference area from changing during the recording process. As a result, it is possible to avoid situations where each setting area Ar moves out of the reference area, preventing the recording of each setting image from being properly captured.

[0158] The control processing performed to suppress changes in the reference area is not limited to the above processing, and may also include, for example, processing to notify the user of suggested information regarding changes in the reference area. For example, a message prohibiting optical zoom may be displayed on the information display screen G3, or played as audio. In addition, as shown in Figure 18, a message prohibiting movement of the shooting device 10 (for example, panning) or a message recommending that the shooting device 10 be fixed using a tripod or other mount may be displayed.

[0159] Furthermore, the control process may be performed before the reference area is determined (i.e., before the recording start instruction is received), for example, within the setting process. In this case, it is possible to prevent the reference area from changing while multiple setting areas Ar are being set within the reference area.

[0160] [Regarding the change process] During the recording process, if predetermined conditions are met, the control unit 46 performs a modification process. The modification process is a process of performing modification processing related to changing the number of images displayed in the setting area Ar (i.e., the setting image). By performing the modification processing, the number of setting images displayed can be dynamically increased or decreased during the recording process. The following describes the processing flow during the recording process, referring to the flowchart shown in Figure 25. For illustrative purposes, Figures 26-31, which will be referenced later, omit the grayed-out display of images other than those within the marker f.

[0161] During the recording process, the control unit 46 monitors the reference video and determines whether the number of subjects in the reference video has increased (S031). If the number of subjects in the reference video has increased, the control unit 46 performs a modification process (S032). In this modification process, a modification process is performed to increase the number of displayable settings videos.

[0162] In the modification process in step S032, a setting area Ar is added in accordance with the increase in the number of subjects in the reference video, and the setting video of the added setting area Ar is newly displayed on the display. As a result, the number of setting videos displayed increases. For example, as shown in Figures 26 and 27, if the number of subjects in the reference video increases from 3 to 4, the number of setting videos displayed (i.e., the number of videos displayed on subscreen G2) increases from 3 to 4.

[0163] Furthermore, if a setting area Ar is added due to an increase in the number of subjects in the reference video, the number of markers f in the reference video displayed on the main screen G1 will increase accordingly by the amount of the added setting area Ar (see Figure 27).

[0164] Furthermore, in the above embodiment, the setting area Ar is set at the position of a new subject in the reference image, but this is not the only option. For example, a candidate setting area Ac may be set instead of setting area Ar. In that case, a marker (specifically, the candidate marker K mentioned above) indicating the position of candidate Ac is displayed in the reference image. When the user performs an operation such as tapping on candidate marker K, this triggers the addition of setting area Ar at the position of candidate Ac. As a result, the setting image for the added setting area Ar is newly displayed, increasing the number of setting images displayed.

[0165] Alternatively, instead of setting a setting area Ar or its candidate Ac at the location of a new subject in the reference video, suggestion information regarding the addition of a setting area Ar (i.e., changing the number of setting videos displayed) may be output. For example, a message suggesting that the number of setting videos displayed be increased by adding a setting area Ar at the location of a new subject may be displayed on the information display screen G3 or played via audio, etc. Thus, as a change process, a process of outputting suggestion information regarding the change in the number of setting videos displayed may be executed.

[0166] In the flow shown in Figure 25, the control unit 46 monitors the reference video during the recording process and determines whether the moving area Am in the reference area overlaps with another set area Ar (S033). If the moving area Am overlaps with the other set area Ar, or is about to overlap, as shown in Figure 28, the control unit 46 outputs a warning message or a warning sound (S034). This is because when the moving area Am overlaps with the other set area Ar, the subject in one area may be behind the subject in the other area and not be visible.

[0167] The system can inform the user of a situation where the moving area Am is overlapping with or about to overlap with another setting area Ar by outputting a warning message or the like. Upon receiving the warning, the user can take appropriate measures (for example, resetting setting area Ar). As one such measure, the user can request a change through screen operation or the like (S035), at which point the control unit 46 will perform a change process (S036). In this change process, a change process is performed to reduce the number of setting images displayed.

[0168] In the modification process in step S036, the overlapping movement area Am and other setting areas Ar are updated into a single setting area Ar, and the updated setting image of the setting area Ar is displayed on the screen. For example, the images of the movement area Am and the other setting area Ar, which were displayed separately as in Figure 28, are integrated into a single image containing the respective subjects of these two images, as in Figure 29. As a result, the number of setting areas Ar is reduced by one, and the number of setting images displayed is also reduced by one.

[0169] Furthermore, it is preferable that the size and position of the updated setting area Ar be set so as to accommodate the subjects of the two videos mentioned above. In addition, the identification information of the marker f indicating the updated setting area Ar may be inherited from the identification information of the marker f indicating either of the two areas before the update (i.e., the moving area Am and the other setting area Ar). Alternatively, a new identification number may be assigned to the updated setting area Ar.

[0170] In the flow shown in Figure 25, the control unit 46 periodically determines whether each of the multiple setting areas Ar satisfies the termination condition during the recording process (S037). The termination condition is a criterion for determining whether or not to terminate the display of the video (setting video) in the setting area Ar, and examples include the following conditions (1) to (3). Cancellation condition (1): At least a portion of the setting area Ar extends beyond the reference area due to optical zoom or the like. Cancellation condition (2): If there is no change in the setting image of setting area Ar for a certain period of time, or if there is no subject in the setting image. Cancellation condition (3): The setting area Ar has not been selected as a recording area for a certain period of time or longer. Furthermore, the conditions for cancellation are not limited to the above conditions (1) to (3), and other conditions may also be included as conditions for cancellation.

[0171] Then, if there is a setting area Ar that satisfies any of the above termination conditions (1) to (3), the control unit 46 performs a modification process (S038). In this modification process, a modification process is performed to reduce the number of setting images to be displayed.

[0172] In the change process in step S038, the display of the setting video for setting area Ar that satisfies the cancellation condition is stopped. For example, in Figure 26, if setting area Ar, where the identification information of marker f is "2", satisfies the cancellation condition, the display of the setting video for that setting area Ar is stopped (it disappears from subscreen G2), as shown in Figure 30. As a result, the number of setting videos displayed decreases by one. Furthermore, for setting area Ar (hereinafter referred to as the "cancellation area") where the display of the setting video has been discontinued, the display of the indicator f indicating the location of the cancellation area on the main screen G1 is also discontinued, as shown in Figure 30.

[0173] Incidentally, instead of immediately stopping the display of the setting video in setting area Ar that satisfies the cancellation conditions, suggested information regarding stopping the display of the setting video (in other words, changing the number of setting videos displayed) may be output. For example, a message recommending the stopping of the display of the setting video in setting area Ar that satisfies the cancellation conditions may be displayed on the information display screen G3, or played via audio, etc. During this time, a different video (a video different from the setting video) may be displayed on the sub-screen G2 instead of the setting video in setting area Ar that satisfies the cancellation conditions.

[0174] When the display of the cancellation video setting video is stopped, the cancellation area is stored in the internal memory 50 of the shooting device 10, etc., while the recording process is being carried out. The period during which the cancellation area is stored may be until the end of shooting, or until the power of the shooting device 10 is turned off.

[0175] Furthermore, when the display of the cancellation area setting video is stopped, operation icons such as the back button Bt9 are displayed on the main screen G1, as shown in Figure 30. The user can, for example, tap the back button Bt9 to instruct the system to redisplay the cancellation area setting video. When the user instructs the system to redisplay the video (S039), the control unit 46 performs a modification process (S040). In this modification process, a modification process is executed to redisplay the stored cancellation area setting video based on the user's redisplay instruction.

[0176] The change process in step S040 causes the video from the canceled area to be displayed again on sub-screen G2, and the number of displayed setting videos returns to the number before the display of the setting videos in the canceled area was canceled. Furthermore, the video footage of the previously canceled area that can be redisplayed is not limited to the video of the most recent canceled area (the area where the display of the video footage was most recently canceled). For example, it may be possible to redisplay video footage of several canceled areas from the past during the current shooting period. In that case, the shooting device 10 stores the video footage of several canceled areas from the past, and the user specifies which canceled area's video footage to redisplay and instructs the device to redisplay the video.

[0177] In the flow shown in Figure 25, the control unit 46 monitors the reference video during the recording process and determines whether the number of subjects in the reference area exceeds the upper limit of the number of set areas Ar (S041). If the number of subjects exceeds the upper limit of the number of set areas Ar, the control unit 46 performs a modification process (S042). In this modification process, modification processing related to reducing the number of displayed set video is performed. In step S042, the modification process displays a message on the information display screen G3, or plays an audio message, indicating that the number of subjects exceeds the upper limit of the number of setting areas Ar, and recommending that the number of setting videos displayed be reduced. The user then reviews the settings of setting areas Ar and deletes unnecessary setting areas Ar, such as the setting area Ar that has not been selected as a recording area for the longest period. As a result, the number of setting videos displayed is reduced by the number of setting areas Ar deleted by the user.

[0178] The series of steps S031 to S042 described above are repeated during the recording process and end when the recording process is completed (S043). As described above, in this embodiment, a modification process is performed in response to changes in the reference video (e.g., the addition of a subject) and the fulfillment of the cancellation condition, and the number of set videos displayed is changed as appropriate. This allows the user to easily check the number of set videos appropriate to the situation at that time, and to omit checking videos in setting area Ar and other areas that are unlikely to become recording areas. As a result, convenience for the user is improved.

[0179] <<Other Embodiments>> The embodiments described above are specific examples given to clearly illustrate the display method of the present invention, and are merely examples; other embodiments may also be conceivable. In the embodiment described above, the shooting device 10 is used as a display device to display various images, but the invention is not limited to this. For example, the display device may consist of the shooting device 10, a camera controller connected to the shooting device 10 by wire or wireless, and an external display. In that case, the camera controller functions as the control unit 46, causing the shooting device 10 to capture a reference image and display various images on the external display. The camera controller may also perform tasks such as setting the setting area Ar, selecting and re-selecting (switching) the recording area, and recording the recorded image.

[0180] Furthermore, in the above embodiment, control processing is performed in the control step during the video display flow to suppress changes in the reference area. However, the system is not limited to this, and as control processing related to changes in the reference area, processing to encourage changes in the reference area may also be performed. In this case, the reference area can be adjusted in accordance with changes in the reference video (for example, the movement of a subject in the reference video).

[0181] To explain in more detail, for example, suppose the recorded video is zoomed in using electronic zoom during the recording process. If the zoom magnification exceeds a predetermined value, the resolution of the recorded video may fall below a certain value, making it difficult to obtain the desired image quality. In such cases, as shown in Figure 31, information prompting a change in the reference area may be displayed on the information display screen G3 by performing optical zoom instead of electronic zoom. Alternatively, for example, the control unit 46 may move the zoom drive unit to forcibly perform optical zoom and automatically change the reference area so that the resolution is above a predetermined value. When the reference area changes due to optical zoom, the relative position of each setting area Ar with respect to the reference area changes. In this case, it is preferable for the control unit 46 to calculate the amount of change in the reference area and adjust the position and size of each setting area Ar based on the calculated amount of change. This ensures that the relative position of each setting area Ar with respect to the reference area is maintained before and after zooming.

[0182] Furthermore, during the recording process, the moving area Am may follow the movement of the subject and reach the edge of the reference area, or at least a part of the moving area may extend beyond the reference area. In such cases, information prompting the user to move the shooting device 10 so that the moved area Am is contained within the reference area (i.e., change the reference image) may be displayed on the information display screen G3. Here, if the user changes the reference area by moving the camera 10 during the recording process, it is preferable that a known detection sensor such as a gyro sensor provided in the camera 10 detects the movement of the camera 10 (e.g., displacement). Then, it is preferable that the control unit 46 adjusts the position and size of each setting area Ar based on the detected movement of the camera 10. This ensures that the relative position of each setting area Ar with respect to the reference area is adjusted according to the changed reference area. In this case, setting areas Ar that have moved outside the reference area due to the change in the reference area may be automatically deleted.

[0183] Furthermore, in the above embodiment, the recorded video is assumed to be a video, that is, a collection of multiple frame images captured continuously at a constant frame rate. However, the recorded video is not limited to a video; it may also be a still image. For example, the control unit 46 displays a reference image as a live view image on the main screen G1 and sets multiple setting areas Ar within the reference area. The user selects a recording area Ap from among the multiple setting areas Ar, and the control unit 46 displays multiple setting images, including the recording image, on the sub-screen G2. Then, when the user issues a recording command by operating the release button 26 or the like, the control unit 46 may record the recording image as a still image on the recording medium.

[0184] Furthermore, in the above embodiment, when a subject is detected in the reference image, the setting area Ar is automatically set at the position of the detected subject. However, the system is not limited to immediately setting the setting area Ar at the position of the detected subject; a candidate setting area Ac may be set first at the position of the detected subject. In that case, a candidate marker K may be displayed at the position of candidate Ac, and the setting area Ar may be set at the position of candidate Ac when the user performs a predetermined operation on the candidate marker K.

[0185] Furthermore, in the above embodiment, the display of the setting video for setting area Ar that satisfies the termination condition is terminated, while it is stored in the shooting device 10 during the recording process. However, it is not limited to this, and setting area Ar that satisfies the termination condition may be deleted, and the video of that area may not be stored. In that case, it is preferable to display suggested information regarding the deletion of setting area Ar on the information display screen G3 before deleting the setting area Ar that satisfies the termination condition. Then, a separate screen (not shown) for selecting whether or not to delete the setting area Ar may be displayed, and the setting area Ar that satisfies the termination condition may be deleted after confirming the user's intention regarding whether or not to delete it.

[0186] Furthermore, although the imaging device was a digital camera in the above-described embodiment, it may also be a video camera, a mobile phone with an imaging optical system, a smartphone, or a tablet device. In addition, the imaging lens may be a lens unit that is externally attached to the imaging optical system of the mobile device. [Explanation of symbols]

[0187] 10 Imaging device 12. Main unit of the imaging device 13 Mount 14. Shooting lens 17 Optical Unit 18,19 Optical equipment 20 aperture 21. Zoom drive unit 22 Focus drive unit 23 Aperture drive unit 26 Release button 28 displays 29 Electronic Viewfinder 30 Operation Buttons 36 Touch Panel 38 shutters 40 Image sensors 42 pixels 44 Analog signal processing circuits 46 Control Unit 47 Controllers 48. Video Processing Section 50 internal memory 52 card slots 54 memory cards 56 Buffer memory 61 Camera Controller 62 External displays 101 First UI screen 102 Second UI screen 103 Third UI screen 104 The fourth UI screen 105 Input screen AC candidate Am moving area Ap recording area Ar setting area As, At shooting area Bt1~Bt8 buttons Bt9 Back button f,fx sign G1 Main Screen G2, G2x Sub-screen G3 information display screen K candidate indicator L1 optical axis T Table Data

Claims

1. A display method for displaying images captured by a camera, A setting step involves setting multiple setting areas within a reference area which is the shooting area of ​​the reference image and corresponds to the shooting angle of the shooting device, A selection step in which the recording area, which is the area of ​​the recorded video to be recorded, is selected from the above-mentioned multiple setting areas, After performing the selection step, a switching step is performed to re-select the recording area from the plurality of setting areas and switch the recording area, A recording step of recording the aforementioned recorded video onto a recording medium, A display step of displaying multiple setting images which are images of the multiple setting areas, The system includes a control step that performs a control process for changing the reference area, at least after determining the reference area, A display method comprising the control process, which includes a process to maintain a state in which the optical zoom function of the shooting device is restricted and the reference area is fixed so that the shooting angle of view is maintained while the recording process is being performed.

2. The display method according to claim 1, wherein the process of maintaining the state in which the optical zoom function is restricted locks the zoom drive unit corresponding to the optical zoom function.

Images