Imaging apparatus and control method for the same

Abstract

To provide an imaging apparatus capable of flexible automatic recording and control method for the same.SOLUTION: The imaging apparatus detects a specific subject from an image obtained by using an image sensor. A control circuit of the imaging apparatus controls the imaging apparatus so as to start recording an image when the state of the specific subject meets a predetermined recording start condition and terminate recording the image when the state of the specific subject meets a predetermined recording termination condition.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus and a control method thereof. [Background technology]

[0002] 2. Description of the Related Art Conventionally, imaging devices are known that automatically record a moving subject that is worthy of attention when it reaches a predetermined position within a shooting range (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-41299 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, it is not possible to automatically record the target subject at a position other than the specific position. Therefore, it is not possible to change the angle of view and the shooting direction after the specific position is set. In addition, it is not possible to automatically record the target subject according to its state (e.g., a specific movement, posture, pose, etc.).

[0005] In view of the problems with the conventional techniques, one aspect of the present invention provides an imaging apparatus capable of flexible automatic recording and a control method thereof. [Means for solving the problem]

[0006] The above-mentioned object can be achieved by an imaging device characterized by having a detection means for detecting a specific subject from an image obtained using an imaging element, and a control means for controlling the imaging device so as to start recording of an image when the state of the specific subject satisfies a predetermined recording start condition, and to end recording of the image when the state of the specific subject satisfies a predetermined recording end condition. Effect of the Invention

[0007] According to the present invention, it is possible to provide an imaging apparatus capable of flexible automatic recording and a control method thereof. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the appearance of an imaging device according to an embodiment and a movable direction of a lens barrel; [Diagram 2] FIG. 1 is a block diagram showing an example of a functional configuration of an imaging apparatus according to an embodiment; [Diagram 3] FIG. 1 is a diagram showing a communication method between an imaging device and an external device according to an embodiment; [Figure 4] FIG. 1 is a block diagram showing an example of a functional configuration of an external device according to an embodiment; [Diagram 5] 1 is a flowchart showing the operation of a control circuit according to an embodiment of the present invention; [Figure 6] 1 is a flowchart showing the operation of a control circuit according to an embodiment of the present invention; [Figure 7] 1 is a flowchart showing the operation of a control circuit according to an embodiment of the present invention; [Figure 8] FIG. 1 is a diagram for explaining a specific example of motion detection in a surfing scene; [Figure 9] FIG. 1 is a diagram for explaining a specific example of motion detection in a snowboarding scene; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described in detail below based on its exemplary embodiments with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. In addition, although multiple features are described in the embodiments, not all of them are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numbers are used for the same or similar configurations, and duplicated explanations are omitted.

[0010] In the following embodiment, the present invention will be described with respect to a case where the present invention is implemented in an imaging device as an example of an image processing device. However, the present invention can be implemented in any electronic device having an imaging function, or any electronic device capable of controlling an imaging device. Such electronic devices include video cameras, computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, robots, drones, and drive recorders. These are merely examples, and the present invention can also be implemented in other electronic devices.

[0011] <Configuration of imaging device> 1(a) is a perspective view showing an example of the appearance of an imaging device 101 as an example of an image processing device according to an embodiment of the present invention. It is assumed here that the imaging device 101 is a PTZ camera that has electric pan, tilt, and zoom functions and is capable of unmanned recording. However, none of the electric pan, tilt, and zoom functions or the unmanned recording function is essential.

[0012] The imaging device 101 has a lens barrel 102 and a fixed part 103 that supports the lens barrel 102. The fixed part 103 may be fixed to an installation location with a screw or the like, or may simply be placed without being fixed. The lens barrel 102 is attached to the fixed part 103 so that it can be panned and tilted. FIG. 1(b) shows the rotation direction of the lens barrel 102 when the Y axis is the vertical direction, the Z axis is the optical axis direction of the imaging optical system that the lens barrel 102 has, and the X axis is a direction perpendicular to the Y axis and the Z axis. The origin O in FIG. 1(b) is the three-dimensional position of the imaging device 101.

[0013] The lens barrel 102 has a tilt rotation unit 104, which is a motor drive mechanism that rotates the lens barrel 102 in the pitch direction shown in Fig. 1(b) relative to the fixed part 103. The lens barrel 102 also has a pan rotation unit 105, which is a motor drive mechanism that rotates the lens barrel 102 in the yaw direction shown in Fig. 1(b) relative to the fixed part 103.

[0014] Fixed portion 103 is provided with angular velocity sensor 106 and acceleration sensor 107 for detecting vibrations of imaging device 101. By driving tilt rotation unit 104 and pan rotation unit 105 so as to cancel out vibrations of imaging device 101 detected based on the outputs of angular velocity sensor 106 and acceleration sensor 107, it is possible to correct the shake and tilt of lens barrel 102. Note that angular velocity sensor 106 and acceleration sensor 107 may be, for example, a 6-axis inertial sensor mounted in the same package.

[0015] 2 is a block diagram showing an example of the internal configuration of the imaging device 101. The control circuit 221 has one or more processors (e.g., a CPU, a GPU, a microprocessor, an MPU, etc.) capable of executing programs, and a system memory (e.g., a DRAM, an SRAM, etc.). The processor (hereinafter, for convenience, referred to as a CPU) controls the operation of each unit and realizes the functions of the imaging device 101 by loading a program stored in, for example, the non-volatile memory 214 into the system memory and executing it.

[0016] The non-volatile memory 214 may be electrically rewritable, such as an EEPROM, and stores programs executable by the CPU of the control circuit 221, settings and constants required for the operation of the image capture device 101, and the like.

[0017] The lens barrel 102 has a zoom unit 201 and a focus unit 203 that constitute a photographing optical system, and an imaging section 206 that converts an optical image formed by the photographing optical system into a digital image signal. The zoom unit 201 is driven by a zoom drive circuit 202 and adjusts the angle of view of the photographing optical system. The focus unit 203 has a focus lens driven by a focus drive circuit 204 and adjusts the focal distance of the photographing optical system.

[0018] The imaging unit 206 has an imaging element and an A / D converter. The imaging element may be, for example, a known CCD or CMOS color image sensor having a primary color Bayer array color filter. The imaging element has a pixel array in which a plurality of pixels are arranged two-dimensionally, and a peripheral circuit for reading out a signal from each pixel. Each pixel accumulates a charge according to the amount of incident light by photoelectric conversion. By reading out a signal having a voltage according to the amount of charge accumulated during the exposure period from each pixel, the optical image formed on the imaging surface by the photographing optical system is converted into a pixel signal group (analog image signal). The analog image signal is converted into a digital image signal by the A / D converter, and then input from the imaging unit 206 to the image processing circuit 207.

[0019] In this embodiment, each pixel of the image sensor has one microlens and two photoelectric conversion regions, and a parallax image pair can be generated in one shooting. The parallax image pair is used to generate an image signal pair used for AF of the phase difference detection method, or to generate a depth image (depth map) in which pixel values ​​indicate depth information or distance information. Note that an image sensor having such a configuration and a method of generating an image signal pair or a depth map using a parallax image pair are well known, so detailed description will be omitted.

[0020] The image processing circuit 207 can be realized by, for example, an arithmetic circuit equipped with a plurality of ALUs (Arithmetic and Logic Units), an application specific integrated circuit (ASIC), a digital signal processing circuit (DSP), etc. The image processing circuit 207 applies predetermined image processing to the digital image signal output from the imaging unit 206 to generate at least one of image data for recording and image data for display. The image data generated by the image processing circuit 207 may be either moving image data or still image data. For example, the moving image data and still image data may be generated in parallel or in a time-division manner.

[0021] The image processing applied by the image processing circuit 207 can include, for example, pre-processing, color interpolation processing, correction processing, detection processing, data processing, evaluation value calculation processing, special effect processing, and the like. Pre-processing may include signal amplification, reference level adjustment, defective pixel correction, etc. Color interpolation processing is performed when an image sensor is provided with a color filter, and is a process for interpolating values ​​of color components that are not included in the individual pixel data that constitutes the image data. Color interpolation processing is also called demosaic processing. The correction processing can include white balance adjustment, gradation correction, correction of image degradation caused by optical aberration of the imaging optical system (image restoration), correction of the effect of peripheral light falloff of the imaging optical system, color correction, and the like. The detection process may include detection of feature regions (for example, face regions or human body regions) and their movements, person recognition processing, and the like. Data processing may include processes such as area extraction (trimming), synthesis, scaling, encoding and decoding, header information generation (data file generation), etc. Data processing also includes the generation of image data for display or image data for recording. The evaluation value calculation process can include processes such as generation of a signal and evaluation value used in automatic focus detection (AF) and generation of an evaluation value used in automatic exposure control (AE). The special effects processing may include adding a blur effect, changing color tones, relighting, and the like. Note that these are examples of processes that the image processing circuit 207 can apply, and do not limit the processes that the image processing circuit 207 can apply.

[0022] The image recording circuit 208 applies processing to the image data generated by the image processing circuit 207 as necessary according to the purpose of the image data and the recording (transmission) destination. Note that encoding and image data file generation may be performed by the image recording circuit 208 instead of the image processing circuit 207. The image recording circuit 208 outputs the image data via the control circuit 221 to a recording (transmission) destination such as the memory 213 or the video output circuit 215.

[0023] The lens barrel rotation drive circuit 205 drives the tilt rotation unit 104 and the pan rotation unit 105 under the control of the control circuit 221 .

[0024] The shake detection circuit 209 has an angular velocity sensor (gyro sensor) 106 that detects angular velocity and an acceleration sensor 107 that detects acceleration, for example, in three axial directions of the image capture device 101. The shake detection circuit 209 detects, for example, angular velocity around the X-, Y-, and Z-axes and acceleration in the X-, Y-, and Z-axis directions shown in FIG. 1(b) based on the output signals of the angular velocity sensor 106 and the acceleration sensor 107. Then, the shake detection circuit 209 determines tilt and pan drive amounts for suppressing shake of the image capture device 101 based on the detected angular velocity and acceleration, and outputs the determined values ​​to the control circuit 221.

[0025] The audio input circuit 211 has a plurality of microphones that output audio around the imaging device 101 as analog audio signals, and an A / D converter that converts the analog audio signals into digital audio signals (audio data). The audio processing circuit 212 applies signal processing such as noise reduction processing to the audio data input from the audio input circuit 211. The audio data processed by the audio processing circuit 212 is stored in the memory 213 via the control circuit 221. The audio data stored in the memory 213 is used by the image processing circuit 207 and / or the image recording circuit 208 when generating moving image data.

[0026] Furthermore, the voice input circuit 211 can estimate the direction of the main sound source based on input from multiple microphones. Information regarding the direction of the main sound source can be used for subject search and automatic recording, which will be described later. Furthermore, the voice processing circuit 212 can recognize specific voice commands. The voice processing circuit 212 may be capable of recognizing not only voice commands registered in advance, but also voice commands registered by the user.

[0027] The audio processing circuit 212 can recognize the state of the shooting scene based on the audio signal input from the microphone. For example, it can recognize states such as "cheering is being heard," "applause is being heard," and "a specific sound is being emitted." This recognition can be realized by a known method, for example, using a neural network that has been trained in advance for each state to be recognized. When the audio processing circuit 212 recognizes a voice command or the state of the scene, it outputs the recognition result to the control circuit 221.

[0028] The memory 213 is used as a working memory for the image processing circuit 207 and the recording / reproducing circuit 218, and as a buffer memory for image data and audio data.

[0029] The recording and playback circuit 218 records image data for recording generated by the image processing circuit 207, control data related to shooting, and the like onto a recording medium 219. Note that audio data may be recorded separately. The recording and playback circuit 218 also reads out data recorded onto the recording medium 219. The read out data is stored in the memory 213 by the control circuit 221. When encoded image data is read out, it is decoded by the image processing circuit 207. The image processing circuit 207 outputs the decoded image data to the video output circuit 215 and the audio data to the audio output circuit 216.

[0030] The recording medium 219 is, for example, a memory card, but any other type of recording medium can be used. There may be multiple recording media 219.

[0031] The power supply circuit 210 supplies power for operating the control circuit 221 .

[0032] The audio output circuit 216 has a speaker and outputs a preset sound, for example, during recording.

[0033] The LED control circuit 222 has one or more light emitting diodes (LEDs), and controls the on / off and / or lighting color of the LEDs in a preset pattern, for example, during recording.

[0034] The video output circuit 215 is, for example, a video output terminal, and transmits an image signal to display a video on a connected external display, etc. The audio output circuit 216 and the video output circuit 215 may be combined into one terminal, such as an HDMI (High-Definition Multimedia Interface) (registered trademark) terminal.

[0035] The environmental sensor 217 is one or more sensors that acquire information about the environment of the imaging device 101. Specific examples of the environmental sensor 217 include, but are not limited to, a temperature sensor, a humidity sensor, an air pressure sensor, an illuminance sensor, and an ultraviolet ray (UV) sensor. The environmental sensor 217 performs measurement at a predetermined cycle, for example, and outputs a value representing the measurement result to the control circuit 221. The control circuit 221 can use the value obtained from the environmental sensor 217 and its changes to determine the shooting scene and to determine whether to execute automatic recording.

[0036] The communication circuit 220 enables the imaging device 101 to transmit and receive data to and from an external device. The communication circuit 220 has a transmission / reception circuit according to a communication standard to be supported. The communication circuit 220 may support any type and number of wired and / or wireless communication standards. Representative examples of communication standards include USB, wireless LAN, 3GPP standards (e.g., 3G, 4G, 5G), Bluetooth (registered trademark), and the like.

[0037] By communicating with the external device through the communication circuit 220, the external device can remotely control the imaging device 101. The remote control can include starting and stopping recording, panning, tilting, and zooming, changing settings of the imaging device 101, and the like. In addition, the imaging device 101 can exchange various parameters related to machine learning used by the audio processing circuit 212 to recognize the state of a scene with the external device through the communication circuit 220.

[0038] The operation unit 223 is a general term for input devices that can be operated by the user, such as a power button and a shooting button.

[0039] <Wireless communication system> 3 is a schematic diagram showing an example of the configuration of a wireless communication system in which an imaging device 101 and an external device 301 are connected. Here, the external device 301 is a smartphone, and is capable of wireless LAN communication 302 and Bluetooth communication 303 with the imaging device 101 (communication circuit 220). Note that the imaging device 101 and the external device 301 may communicate according to other standards. Here, it is assumed that the imaging device 101 and the external device 301 are capable of communication using two or more communication standards with different power consumption and / or communication distances. It is assumed that the Bluetooth communication 303 complies with the Bluetooth Low Energy (BLE) standard. Communication according to the BLE standard consumes less power and has a shorter communication distance than communication according to the wireless LAN standard.

[0040] <External device configuration> 4 is a block diagram showing an example of a functional configuration of an external device (smartphone) 301. The external device 301 has a wireless LAN control circuit 401, a BLE control circuit 402, a public line control circuit 406, and a packet transmission / reception circuit 403 as communication circuits for communicating with other devices.

[0041] The wireless LAN control circuit 401 includes an antenna, a transmission / reception circuit, a communication control circuit, and the like for performing communication conforming to the wireless LAN standard. The BLE control circuit 402 has an antenna, a transmission / reception circuit, a communication control circuit, and the like for performing communication in accordance with the BLE standard. The public line control circuit 406 has an antenna, a transmission / reception circuit, a communication control circuit, and the like for performing communication compliant with the 3GPP standards (3G, 4G, 5G, etc.).

[0042] The packet transmission / reception circuit 403 executes packet processing by communication through the wireless LAN control circuit 401, the BLE control circuit 402, and the public line control circuit 406. Note that, in this embodiment, the external device 301 performs packet communication with other devices, but other communication methods such as circuit switching may be used.

[0043] The control circuit 411 has one or more processors (e.g., CPU, GPU, microprocessor, MPU, etc.) capable of executing programs, and a system memory (e.g., DRAM, SRAM, etc.). The processor (hereinafter, for convenience, referred to as CPU) loads a program stored in the storage circuit 404 into the system memory and executes it, thereby controlling the operation of each part and realizing the functions of the external device 301.

[0044] The sensor 412 is one or more sensors that acquire information regarding the movement, posture, environment, etc. of the external device 301. Specific examples of the sensor 412 include, but are not limited to, a proximity sensor, an angular velocity sensor, an acceleration sensor, a temperature sensor, a humidity sensor, an air pressure sensor, an illuminance sensor, and an ultraviolet ray (UV) sensor. The sensor 412 performs measurement at a predetermined period, for example, and outputs a value representing the measurement result to the control circuit 411.

[0045] The storage circuit 404 may be, for example, an electrically rewritable non-volatile memory. The storage circuit 404 stores programs executable by the CPU of the control circuit 411, setting values ​​and constants necessary for the operation of the external device 301, user data, and the like. In general, in a smartphone, an application program is executed while an operating system (OS) is running, thereby providing a function specific to the application. In this embodiment as well, the functions to be described later as those executed by the external device 301 are provided by an application for providing the function while using the function of the OS as necessary.

[0046] The power supply circuit 410 supplies power for operating each part of the external device 301, including the functional blocks shown in FIG. The display device 407 is, for example, a liquid crystal display (LCD) or an organic light emitting display (OLED). The display device 407 displays a graphical user interface (GUI) provided by the OS and applications, various information, user data, etc. For convenience, the display device 407 also includes a speaker.

[0047] The operation member 408 has one or more input devices that the external device 301 has to receive input from a user. For example, a fingerprint input device, a touch panel, a volume button, a power button, etc. are included in the operation member 408. The touch panel may be incorporated in the display device 407.

[0048] The voice input / processing circuit 409 acquires a voice signal, for example, from a microphone provided in the external device 301, and applies predetermined processing to the acquired voice signal. The predetermined processing may be, for example, voice recognition processing. In this case, the voice input / processing circuit 409 outputs the voice recognition result to the control circuit 411. If the voice recognition result can be interpreted as a voice command, the control circuit 411 executes processing according to the voice command.

[0049] The voice command may be for remotely controlling the imaging device 101. When the recognition result includes a command for remotely controlling the imaging device 101, the control circuit 411 transmits the command to the imaging device 101 via the wireless LAN control circuit 401. Note that a user-specific voice command may be generated using an application running on the external device 301. The generated voice command is transmitted to the imaging device 101 and registered, so that it can be used as a voice command that can be recognized by the voice processing circuit 212.

[0050] The GPS (Global Positioning System) receiver 405 receives signals transmitted from GPS satellites, and estimates the current position (longitude and latitude information) of the external device 301 based on the received signals. Note that the current position of the external device 301 may be estimated using other methods such as WPS (Wi-Fi Positioning System).

[0051] When the estimated current location is within a preset geographical range or when the location information has changed by a predetermined threshold or more, the control circuit 411 transmits the location information to the imaging device 101 via the BLE control circuit 402. The location information of the external device 301 can be used for automatic recording and automatic editing, which will be described later.

[0052] In this way, the external device 301 exchanges data with the imaging device 101 through wireless LAN communication 302 and Bluetooth (BLE) communication 303. For example, data such as audio signals, image signals, compressed audio signals, and compressed image signals are transmitted and received. The external device 301 also transmits to the imaging device 101 operation instructions such as detection values ​​of the sensor 412 and recording, transmits voice command registration data, and performs position detection notification and movement notification based on position information. Learned data used by the audio processing circuit 212 can also be transmitted from the external device 301 to the imaging device 101 and received from the imaging device.

[0053] <Image capture operation> FIG. 5 is a flowchart relating to the operation of the imaging device 101. When the power button of the operation unit 223 is operated in the power-off state, the control circuit 221 starts the startup process in S501. The startup process may be started under other conditions. For example, it may be started remotely by communication from the external device 301 (e.g., BLE communication). It may be started by detecting the movement of the imaging device 101. It may also be started by detecting a voice command instructing startup. When the startup process is executed in response to a remote operation or detection of a specific event, power is supplied from the power supply circuit 210 to components required for detection even in the power-off state.

[0054] In S502, the control circuit 221 acquires various detection values. Here, at least the sensors that detect the movement of the imaging device 101 (angular velocity sensor 106 and acceleration sensor 107 of the shaking detection circuit 209) and the rotation positions of the tilt rotation unit 104 and pan rotation unit 105 are read. The control circuit 221 may further acquire detection values ​​(audio level, direction of sound source, etc.) of the audio processing circuit 212, detection values ​​of the environment sensor 217, etc.

[0055] In S503, the control circuit 221 communicates with the external device 301 as necessary. The control circuit 221 communicates with the external device 301, for example, when communication is requested by the external device 301 or when there is data to be transmitted to the external device 301. Note that, for example, as part of the startup process or as a process performed after the startup process, the procedures required for communicating with the external device 301 (recognizing the external device 301, checking the communication methods supported, establishing communication, etc.) are executed.

[0056] Here, one or both of wireless LAN communication and BLE communication can be established with the external device 301. Also, BLE communication with low power consumption may be established first, and wireless LAN communication may be established as necessary when a large amount of data is communicated or high-speed communication is required. After wireless LAN communication is established, BLE communication may be maintained or disconnected.

[0057] In S503, the control circuit 221 can receive, for example, a remote operation command, voice command registration data, position detection notification, movement notification, and measurement values ​​of a sensor included in the external device 301 from the external device 301. The control circuit 221 can also transmit, for example, voice data, image data, and messages to the external device 301. When communication with the external device 301 ends, the control circuit 221 executes S504.

[0058] In S504, the control circuit 221 determines the mode to be set. Here, the following modes are assumed to be settable in S504, but other modes may be included. (1) Manual recording mode [Mode determination conditions] When an instruction to set the manual recording mode is given via the operation unit 223, or when a command to set the manual recording mode is received from the external device 301, the control circuit 221 decides to set the manual recording mode and executes S506. When it is determined that the mode determination condition is not satisfied, the control circuit 221 executes S507.

[0059] [In-mode processing] In the manual recording mode process of S506, the control circuit 221 controls the operation of the imaging device 101 according to a user instruction (an operation of the operation unit 223 or a command received from the external device 301). Specifically, the control circuit 221 controls the zoom drive circuit 202, the focus drive circuit 204, and the lens barrel rotation drive circuit 205 according to the user instruction to adjust the shooting range and the focal distance of the imaging device 101. The control circuit 221 also executes recording of still images and videos according to the user instruction.

[0060] (2) Scene-specific recording mode [Mode Judgment Condition] When an instruction to set the scene designated recording mode is given via the operation unit 223, or when a command to set the scene designated recording mode is received from the external device 301, the control circuit 221 decides to set the scene designated recording mode and executes S508. When it is determined that the mode determination conditions are not satisfied, the control circuit 221 executes S509.

[0061] The scene-specified recording mode is a shooting mode in which automatic recording is performed when the imaging device 101 determines a specified scene. Therefore, when the scene-specified recording mode is set, a specific scene is also set. Scenes that can be set are scenes that the imaging device 101 can determine. There are no particular limitations on the type of scene, and they may be landscapes (e.g., night scenes, sunsets, etc.), people (e.g., portraits, commemorative photos, etc.), sports (surfing, snowboarding, skateboarding, BMX, disc dogs, show jumping, parkour, athletic meets, etc.), etc.

[0062] [In-mode processing] In the scene designation recording mode process of S508, the control circuit 221 determines the timing of starting and ending recording according to the designated scene, and executes automatic recording of moving images and / or still images.

[0063] (3) Automatic recording mode [Mode Judgment Condition] If it is determined that the predetermined conditions are satisfied with respect to the information that the imaging device 101 can acquire, the control circuit 221 decides to set the automatic recording mode and executes S510. If it is determined that the mode determination conditions are not satisfied, the control circuit 221 executes S502.

[0064] Here, the information that the imaging device 101 can acquire may be one or more of the following, and changes therein: a captured image, a movement of the imaging device 101 detected by the shake detection circuit 209, a value or information obtained from a voice obtained by the voice input circuit 211, a value obtained by the environmental sensor 217, and date and time. It may also be information obtained from the external device 301, such as a position detection notification, a movement notification, and a measurement value of a sensor possessed by the external device 301, or changes therein. Predetermined conditions regarding the information that the imaging device 101 can acquire are registered in the non-volatile memory 214 in advance. The user may also be able to additionally register them. In addition, when it is determined that automatic recording should be performed based on the elapsed time since the transition to the automatic recording mode, past shooting information, and the like, the control circuit 221 may decide to set the automatic recording mode.

[0065] [In-mode processing] In the automatic recording mode process of S510, the control circuit 221 adjusts the shooting direction and shooting range of the imaging device 101 based on information that the imaging device 101 can acquire, and automatically searches for a subject to be recorded. Then, when a subject is found, the control circuit 221 controls the shooting direction and shooting range of the imaging device 101 so as to track the subject. Furthermore, the control circuit 221 executes automatic recording of videos and / or still images if it is determined that the subject and surrounding conditions match predetermined recording conditions.

[0066] Scene determination can be performed in parallel with the subject search. If it is determined that the scene is a predetermined scene, the control circuit 221 may switch the shooting mode to a scene-designated recording mode for the determined scene. If a recording instruction is manually given during operation in the automatic recording mode, the control circuit 221 executes recording in response to the recording instruction.

[0067] <Scene-specified recording mode processing> Next, the scene designation recording mode process in S508 of Fig. 5 will be described in detail with reference to Fig. 6. During execution of the scene designation recording mode, the control circuit 221 executes the following processes while continuously shooting moving images.

[0068] In S601, the control circuit 221 causes the image processing circuit 207 (subject detection means) to execute subject recognition processing on the digital image signal obtained from the imaging unit 206. The image processing circuit 207 applies pre-processing for subject recognition to the digital image signal as necessary, and then detects areas (subject areas) in which a predetermined type of subject is likely to be captured. Then, the image processing circuit 207 outputs the number, position, size, detection reliability, etc. of detected areas for each type of subject to the control circuit 221 as a result of the subject recognition processing.

[0069] For example, when detecting the subject region of a person or animal, the image processing circuit 207 detects the face region (and also the torso region as necessary). In the face detection process, a region in the image data that matches with preregistered feature patterns of human faces and animal faces at a certain level or higher is detected as the subject region. The feature pattern may be, for example, a template of face parts such as the eyes and mouth, or information such as the positional relationship between the parts. The reliability of detection is calculated to be a higher value, for example, the larger the size of the face region and the higher the match with the feature pattern. For example, the posture of the subject can be determined based on the positional relationship between the face region and the torso region.

[0070] For other types of subjects, the subject area can be detected based on the degree of match with pre-registered feature patterns according to the type of subject. For example, in the case of a vehicle subject, feature patterns for recognizing the overall shape of the vehicle or the person driving / operating the vehicle can be used.

[0071] Also, histograms of hue, saturation, etc. can be used as feature patterns. There are also methods for extracting feature objects by using a method of using. For example, a histogram may be generated for each of a plurality of color components for each partial region, and the object region may be detected based on the relationship between the frequency of the color components and the degree of match with a feature pattern registered in advance.

[0072] When multiple object regions are detected for the same type of object, the control circuit 221 can determine, for example, the object region with the highest detection reliability as the main object region.

[0073] Note that the subject area can be detected using any known method, such as machine learning using a neural network trained for each type of subject, so further details will not be provided here.

[0074] Next, in S602, the control circuit 221 calculates a correction amount for correcting image blur caused by the movement of the image capture device 101. The control circuit 221 calculates angle information of the direction adjustable by the tilt rotation unit 104 and the pan rotation unit 105, which indicates a change in attitude (movement) of the image capture device 101, based on, for example, angular velocity and acceleration information obtained from the shake detection circuit 209.

[0075] Then, the control circuit 221 calculates the drive amounts of the tilt rotation unit 104 and the pan rotation unit 105 for canceling the movement of the image capturing device 101 as correction amounts, based on the calculated angle information.

[0076] In S603, the control circuit 221 determines the state of the imaging device 101. Specifically, the control circuit 221 determines the state of the imaging device 101 from one or more of angular velocity and acceleration information obtained from the shake detection circuit 209, information obtained from the environment sensor 217, position information of the imaging device 101 obtained from the external device 301, and the like.

[0077] An example of the state of the image capture device 101 determined in S603 is whether or not it is moving. For example, it can be determined whether or not it is moving based on whether or not the magnitude of the movement of the image capture device 101 obtained from the angular velocity and acceleration information obtained from the shake detection circuit 209 is equal to or greater than a threshold. Alternatively, it may be determined whether or not it is moving based on the amount of change in position information.

[0078] In addition, in S603, other states of the imaging device 101 may be determined. For example, it is possible to determine whether the imaging device 101 is in an indoor or outdoor state based on information obtained from the environment sensor 217 (for example, information obtained from a UV sensor). It is also possible to determine whether the imaging device 101 is in a specific state based on a combination of multiple pieces of information. Based on the determination result in S603, it is possible to switch between subject search processing and motion detection processing, which will be described later.

[0079] In S604, the control circuit 221 performs a subject search process. The control circuit 221 captures an image of a captureable range corresponding to the entire movable range of the tilt rotation unit 104 and the pan rotation unit 105, centered on the position of the imaging device 101 (origin O in FIG. 1(b)), and divides the captureable range into a plurality of regions. Then, the control circuit 221 calculates a search priority (importance level) for each region. The division can be performed, for example, in units of the capture range at a certain focal length (angle of view).

[0080] The control circuit 221 calculates the importance level of each region and determines the region with the highest importance level as the target region. The control circuit 221 then calculates the target angles of the tilt rotation unit 104 and the pan rotation unit 105 for photographing the range including the target region, and then executes S605.

[0081] The importance level can be calculated based on the situation of the subject included in each region. For example, the importance level can be calculated based on the number, size, orientation, detection reliability, facial expression, and personal recognition result of the person of the face region of the person present in the region. Alternatively, the importance level can be calculated by taking into consideration one or more of the following: detection result of a general object, scene discrimination result (blue sky, backlight, evening scene, etc.), the level of the audio signal input from the direction of the target region and / or the recognition result of the audio signal among the audio signals obtained by the audio input circuit 211, the movement of the subject in the region, etc.

[0082] Also, the state of the imaging device 101 determined in S603 may be reflected in the importance level. For example, when it is determined that the imaging device 101 is not moving, the importance level of the area in which the face area (main subject area) of a person registered in advance exists can be increased. This makes it easier to perform subject search centered on the area in which the face area of ​​the person registered in advance exists. For example, by setting a person registered in advance as the user of the imaging device 101, it becomes possible to perform subject search in the periphery of the range in which the user is captured. A specific subject area for which the importance level is increased in this way is called a subject area to be searched.

[0083] As a result, automatic recording is also performed with priority given to the area including the user's face area. Therefore, the user can automatically record an image of himself / herself without being aware of it. In addition, since the area including the user's face area can be searched for by subject search processing, there is no need to precisely adjust the shooting direction when placing the imaging device 101, and it is sufficient to set it facing in an approximate direction. Even if the user moves, the area including the user's face area will continue to be automatically recorded by subject search processing as long as it is within the movable range of the tilt rotation unit 104 and the pan rotation unit 105.

[0084] If the scene does not change, the importance level of each area does not change, so it is possible that only the same area will be photographed. Therefore, if the area with the highest importance level does not change for a predetermined period of time, the importance level of that area may be lowered, or the importance level of the photographed area may be lowered for a certain period of time, so that the area with the highest importance level may be made more likely to change.

[0085] Also, it is not necessary to calculate the importance level each time S604 is executed. For example, when a specific face region is captured as the main subject region, the tracking process of the main subject region may be executed as the subject search process. In this case, the subject region can be tracked by template matching using the detected main subject region as a template without calculating the importance level for each region. For example, when the main subject region is lost, the importance level for each region may be calculated.

[0086] In S605, the control circuit 221 drives the tilt rotation unit 104 and the pan rotation unit 105. Specifically, the control circuit 221 calculates the pan and tilt drive amounts based on the correction amount calculated in S602 and the target angle calculated in S604. The drive amounts can be calculated as multiples of the minimum control unit of the tilt rotation unit 104 and the pan rotation unit 105.

[0087] The control circuit 221 drives the tilt rotation unit 104 and the pan rotation unit 105 by controlling the lens barrel rotation drive circuit 205 based on the calculated pan and tilt drive amounts.

[0088] In S606, the control circuit 221 controls the zoom driving circuit 202 to drive the zoom unit 201. For example, the control circuit 221 can drive the zoom unit 201 based on the state of the subject area included in the area with the highest importance level in S604.

[0089] For example, when the subject region to be searched is a human face region, if the face does not appear in a certain size in the captured image, it cannot be detected as a face region. Therefore, the control circuit 221 controls the zoom driving circuit 202 to zoom in by a predetermined amount when the size of the face region to be searched falls below a predetermined threshold value, so that the size of the face region to be searched becomes equal to or larger than a predetermined size that can be detected as a face region.

[0090] Conversely, if the face area, which is the subject area to be searched for, is too large, the subject area is likely to go out of the shooting range due to the movement of the subject or the image capture device 101. Therefore, the control circuit 221 controls the zoom drive circuit 202 to zoom out by a predetermined amount when the size of the face area to be searched for exceeds a predetermined threshold, so that the face area to be searched for does not exceed an appropriate size. Note that the threshold for zooming in and the threshold for zooming out can be determined separately.

[0091] If the imaging device 101 is not a PTZ camera but an omnidirectional camera using an ultra-wide-angle lens such as a fisheye lens, the imaging range is wide, so the load required for image processing is large. Therefore, the image processing after calculating the importance level for each area (such as the processing related to the automatic recording judgment described later) may be performed by cutting out a part of the area including the area with the highest importance level, thereby reducing power consumption and processing time.

[0092] Once the shooting direction and angle of view have been determined by the processing up to S606, the control circuit 221 can start a pre-recording operation. The pre-recording operation is an operation that constantly stores or records images from the most recent predetermined period (for example, several seconds to about 10 seconds) in the memory 213 or recording medium 219 in the recording standby state.

[0093] Next, in S607, the control circuit 221 executes a motion detection process to determine whether to start or end recording. In the motion detection process, the start or end of recording is determined based on a recording start condition and a recording end condition that are registered in advance in the non-volatile memory 214 according to, for example, a specified shooting scene. This makes it possible to automatically record a scene that is characteristic of the shooting scene. For example, it is possible to automatically record a video or a plurality of still images of a characteristic scene lasting several to several tens of seconds, such as a jump section in snowboarding or a series of riding sections from a take-off in surfing. In the following, in order to explain a case where the motion of a subject related to a particular sport is used as the recording start condition or the recording end condition, the recording start condition is referred to as a recording start operation, and the recording end condition is referred to as a recording end operation.

[0094] On the other hand, since recording is not performed on scenes other than characteristic scenes, it is possible to suppress unnecessary consumption of storage medium capacity and power. Also, when the user of the image capture device 101 automatically records his / her own images, characteristic scenes can be recorded without the user having to instruct the image capture device 101 to start or stop recording using a remote control or the like, which is convenient.

[0095] In the motion detection process, the control circuit 221 can detect a predetermined recording start motion, a recording end motion, and a motion during recording (a recording condition). The motion during recording corresponds to a state between the recording start motion and the recording end motion, and specifies the state or motion of the subject during the period to be recorded. Registration and detection of the motion during recording are not essential. Details of the motion detection process executed in S607 will be described later with reference to FIG. 7.

[0096] S608 to S616 are operations according to the result of the motion detection process in S607. In S608, the control circuit 221 determines whether or not a recording start operation has been detected, and if it is determined that a recording start operation has been detected, the process proceeds to S609, and if not, the process proceeds to S610. In S609, the control circuit 221 starts automatic recording. This causes image data to be recorded, for example, on the recording medium 219. It is assumed that whether to record moving images or still images during automatic recording, and the recording interval when recording still images, are set in advance. For example, they may be set in advance for each type of scene, or may be set by the user. Thereafter, the control circuit 221 repeats the process from S601.

[0097] In S610, the control circuit 221 determines whether the conditions that an operation during recording has been detected and that recording is not currently in progress are met, and if it is determined that the conditions are met, it executes S611, and if it is not, it executes S612. This condition is met when, even though a scene should be recorded, some factor causes the detection of a recording start operation to fail and recording has not started. Therefore, in S611, the control circuit 221 starts recording, including the pre-recorded image data. This makes it possible to record image data that continues from the pre-recording period. The control circuit 221 then repeats the process from S601.

[0098] In S612, the control circuit 221 determines whether or not a recording end operation has been detected, and if it is determined that it has been detected, the process proceeds to S613, and if not, the process proceeds to S615. In S613, the control circuit 221 executes the same judgment process as in S610, and if it is determined that the condition is met, it executes S614, and if not, it executes S616. This condition is met when, despite the fact that it was a scene that should have been recorded, the detection of the recording start operation failed for some reason, and recording was not started.

[0099] However, because the recording end operation has been detected, in S614 the control circuit 221 does not start recording, unlike in S611. Instead, the control circuit 221 records the pre-recorded image data held in the memory 213 on the recording medium 219, and then executes S616. Note that if the pre-recorded image data is recorded on the recording medium 219 rather than on the memory 213 during pre-recording, the file name or recording directory is changed so that the pre-recorded image data is not overwritten, making it recorded data. The pre-recorded image data includes the period before the recording end operation is detected, that is, the scene that should have been recorded. Therefore, by recording the pre-recorded image data, it is possible to avoid the occurrence of a situation in which the scene that should have been recorded is not recorded at all.

[0100] In S615, the control circuit 221 judges whether the conditions that recording has started and a predetermined time has elapsed since the start of recording are met, and if it is judged that the conditions are met, S616 is executed, otherwise, the process from S601 is repeated. This condition is met when the detection of the recording end operation has failed for some reason and recording is continuing.

[0101] In S616, the control circuit 221 ends the recording of the image data. After that, the control circuit 221 repeats the process from S601.

[0102] The above operations are continuously executed until, for example, the user releases the scene designation recording mode of the image capture apparatus 101 .

[0103] <Motion detection process> Next, the details of the motion detection process in S607 in Fig. 6 will be described with reference to Fig. 7. In S701, the control circuit 221 executes an environment detection process on image data obtained from the imaging unit 206, for example, using the image processing circuit 207. In the environment detection process, objects and shapes according to a specified shooting scene are detected, which are used to detect the motion of the main subject. Specifically, the environment detection process can detect all objects having height relative to a reference plane such as the ground, such as artificial objects such as equipment and tools used in sports, and natural shapes such as wave crests and ground undulations.

[0104] Specific examples of man-made objects include jumps (kickers) used in skiing and snowboarding, hurdles in athletics, obstacles used in equestrian and dog agility, baskets, goals for soccer, etc., volleyball, badminton, tennis nets, etc. These are merely examples, and other objects may be detected according to scenes that can be specified in the scene-specified recording mode.

[0105] There is no particular limitation on the method of environment detection, and any known method can be used. For example, there is a method that utilizes machine learning, such as using a neural network that has been trained in advance for each type of object to be detected, or a method that detects the contour of an object from a luminance distribution. These are merely examples, and other methods may be used. Note that the type of object to be detected is determined in advance according to the type of scene set in the scene-specified recording mode. Also, a parameter set required for detecting an object according to the type of scene is set in the image processing circuit 207. The image processing circuit 207 outputs information (position, size, detection reliability, etc.) of each detected object area for each type of object to the control circuit 221.

[0106] In S702, the control circuit 221 detects the motion of the subject using information on the object detected in S701 and the subject recognized in S601 of Fig. 6. The control circuit 221 determines whether the motion of the subject corresponds to a recording start motion, a recording motion, or a recording end motion that is predetermined according to the scene, based on the state of the subject (e.g., posture) and / or the positional relationship between the subject and the object. Note that if the main subject has been determined among the subjects recognized in S601, such as when the user's face area is determined as the main subject area, the control circuit 221 applies the processes from S702 onwards to the main subject.

[0107] In S703, if it is determined that the subject's movement corresponds to a predetermined movement in S702, the control circuit 221 determines that the movement detection was successful and executes S704. On the other hand, if it is not determined that the subject's movement corresponds to a predetermined movement in S702, the control circuit 221 determines that the movement detection was unsuccessful and executes S705.

[0108] In S705, the control circuit 221 detects the moving direction of the subject using multiple images captured at different times. If the subject cannot be recognized in S601, the control circuit 221 can detect a motion vector for each region between the images to separate the moving object region from the background region, and determine the motion vector corresponding to the moving object region as the moving direction of the subject.

[0109] In S706, the control circuit 221 determines whether the subject's action corresponds to a recording start action, recording action, or recording end action that is predetermined depending on the scene, taking into consideration the state of the subject (e.g., posture) and / or the positional relationship between the subject and an object, as well as the direction of movement of the subject.

[0110] In S707, if it is determined in S706 that the motion of the subject corresponds to a predetermined motion, the control circuit 221 determines that motion detection has been successful and executes S704. On the other hand, if it is not determined in S706 that the motion of the subject corresponds to a predetermined motion, the control circuit 221 determines that motion detection has failed and executes S708.

[0111] In S708, the control circuit 221 detects the movement trajectory of the subject, and determines whether or not the movement trajectory corresponds to any of the movement trajectories corresponding to the recording start operation, the recording operation, and the recording end operation, which are registered in advance according to the scene. The movement trajectory of the subject can be detected by connecting the movement direction of the subject over time, as described in S705. For example, when the similarity between the movement trajectory detected from the image and the registered movement trajectory is equal to or greater than a predetermined threshold, the control circuit 221 can determine that the subject is performing an action corresponding to the registered movement trajectory.

[0112] In S709, if the subject's motion is determined to be a recording start motion, a recording motion, or a recording end motion in S708, the control circuit 221 determines that motion detection was successful and executes S704. On the other hand, if the subject's motion is not determined to be a predetermined motion in S708, the control circuit 221 determines that motion detection was unsuccessful and executes S710.

[0113] In S710, the control circuit 221 acquires depth information for at least the peripheral area of ​​the subject area. The depth information can be acquired, for example, from a depth map obtained based on a pair of parallax images acquired to perform AF for video shooting in a recording standby state. The control circuit 221 then determines whether the action of the subject corresponds to a recording start action, a recording action, or a recording end action that is predetermined according to the scene, taking into consideration, for example, the state of the subject (e.g., posture) and / or the positional relationship between the subject and an object, as well as the depth map.

[0114] For example, when the depth information directly below the subject area is greater (farther) than the depth area of ​​the subject area by a threshold or more, the control circuit 221 can determine that the subject is jumping. For example, the control circuit 221 determines whether or not a combination of the state of the subject that can be determined from the depth information and the state of the subject obtained from the image (e.g., posture) corresponds to a recording start operation, a recording operation, or a recording end operation that is predetermined according to the scene.

[0115] In S712, if the control circuit 221 determines in S711 that the subject's motion is a recording start motion, a recording motion, or a recording end motion, it determines that motion detection was successful and executes S704. On the other hand, if the subject's motion is not determined to correspond to a predetermined motion in S711, the control circuit 221 stores information indicating that the motion detection process failed in S712 and ends the motion detection process.

[0116] On the other hand, in S704, the control circuit 221 saves information indicating the type of the detected operation (recording start operation, recording operation, or recording end operation) as a result of the operation detection process, and ends the operation detection process.

[0117] 7, if a motion cannot be detected in S702 based on the state (e.g., posture) of the subject and / or the positional relationship between the subject and an object, the motion is detected by sequentially considering the subject's movement direction, the subject's movement trajectory, and the depth information of the shooting range. However, the order in which the subject's movement direction, the subject's movement trajectory, and the depth information of the shooting range are considered is not limited to that shown in Fig. 7. Also, in S702, a motion may be detected by considering one or more of the subject's movement direction, the subject's movement trajectory, and the depth information of the shooting range.

[0118] <For surfing> A specific example of the motion detection process when surfing is specified in the scene-specified shooting mode will be described with reference to FIGS.

[0119] Surfing is a sport in which you lie down on a surfboard and paddle with your hands to follow the progress of the wave, stand on the board when the peak of the wave breaks (take off), and ride using the power of the wave. In the scene-specified shooting mode, the camera automatically records the scene from the start of paddling to the end of riding. Figure 8(a) shows the state just before takeoff, and Figure 8(b) shows the state during riding.

[0120] Because the waves generated during paddling are thick, the brightness of the waves is lower than that of the sky and the ocean in the background, as shown in Figure 8(a). Taking advantage of this characteristic, the environment detection (S701) calculates the gradient of the vertical brightness and detects the boundary line in the approximately horizontal direction where the gradient is equal to or greater than a predetermined threshold. This boundary line corresponds to the wave crest (peak).

[0121] Also, since the subject is located vertically below the peak during paddling, a state in which the subject is located vertically below the peak is registered as a recording start operation. Therefore, if it is detected in the operation detection (S702) that the subject is located vertically below the peak, it is determined that a recording start operation has been detected. In addition, a condition that the subject is approaching the image capture device 101 may be added as a condition for the recording start operation. In this case, in S702, the depth information is further taken into consideration to determine whether or not the condition that the subject is approaching the image capture device 101 is met.

[0122] As shown in Fig. 8(b), when the subject takes off, the torso of the subject can be detected by S601. Therefore, as an action during recording, it is registered that a peak has been detected by S701 and that the face of the subject and the moving object are aligned vertically. When it is detected that these conditions are met, it is determined that an action during recording (riding) has been detected.

[0123] In addition, as the recording end action, the disappearance of the waves and / or the transition of the subject from a standing state to a lying state are registered. Therefore, when a peak cannot be detected in S701, or when the torso cannot be detected and the positional relationship between the face and torso cannot be determined in S702, it is determined that the recording end action has been detected.

[0124] <For snowboarding> Next, a specific example of the motion detection process when snowboarding (jump scene on a kicker) is specified in the scene-specified shooting mode will be described with reference to FIGS.

[0125] In scene-specified shooting mode, the scene is automatically recorded from the approach, which corresponds to the entrance of the kicker, through the jump at the lip, which is the jumping-off point, to the landing. Figure 9(a) shows the approach and Figure 9(b) shows the jump at the lip. The arrows also show the movement trajectory.

[0126] In S701, the control circuit 221 detects the edge portion (lip) of the kicker using the luminance gradient, just like in surfing. However, unlike in surfing, the kicker is white because it is covered with snow, whereas the mountains and sky in the background have a lower luminance, so the positive and negative luminance thresholds are reversed from those in surfing.

[0127] In addition, in the case of a kicker, the background is covered with snow, and it may be difficult to detect the lip using only brightness. Therefore, a machine learning method such as segmentation may be used to detect the lip.

[0128] Here, the shooting start action is registered as the subject moving toward the lip and the distance between the subject and the lip being equal to or less than a predetermined threshold value. Therefore, the action is detected taking into consideration the subject's moving direction and movement trajectory.

[0129] If both the direction of movement of the subject is a direction approaching the lip detected in S701 and the distance between the position of the subject area and the lip is less than a predetermined threshold, it can be determined that a recording start operation has been detected.

[0130] Moreover, the motion being recorded is a jump motion. In a jump motion, the subject's movement trajectory becomes parabolic as shown in FIG. 9(b), and thus the parabolic movement trajectory is registered as the motion being recorded. Therefore, if the similarity between the subject's movement trajectory and the registered parabolic movement trajectory is equal to or greater than a threshold, it can be determined that the motion being recorded has been detected. In addition, in order to detect a jump motion, depth information may be further taken into consideration as described above. If the difference between the depth information of the subject and the depth information of the area vertically below the subject is equal to or greater than a predetermined threshold, it can be determined that a jump motion is being performed.

[0131] As the recording end action, a jump action followed by landing is registered. Specifically, when it is detected that the difference between the depth information of the subject and the depth information of the area vertically below the subject has changed from a state equal to or greater than a predetermined threshold to a state less than the predetermined threshold, it can be determined that the recording end action has been detected.

[0132] In this way, the control circuit 221 acquires information necessary to determine whether the conditions for the recording start operation, the recording operation, and the recording end operation registered in advance for each scene are met, and executes the operation determination process. The specific conditions for the recording start operation, the recording operation, and the recording end operation can be experimentally determined and registered for each scene, for example.

[0133] According to this embodiment, the conditions for the recording start operation, the recording operation during recording, and the recording end operation according to the scene are registered in advance, and automatic recording is started when the recording start operation condition is satisfied, and automatic recording is ended when the recording end operation condition is satisfied. By registering the conditions for the recording start operation, the recording operation during recording, and the recording end operation appropriate for the scene, it is possible to provide an image processing device capable of flexible automatic recording.

[0134] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0135] The disclosure of this embodiment includes the following imaging device, imaging device control method, and program. (Item 1) A detection means for detecting a specific subject from an image obtained by using the image sensor; a control means for controlling the imaging device to start recording of the image when a state of the specific subject satisfies a predetermined recording start condition, and to end recording of the image when the state of the specific subject satisfies a predetermined recording end condition; An imaging device comprising: (Item 2) 2. The imaging device according to item 1, wherein the control means controls the imaging device to continue holding the image for a predetermined period of time before starting to record the image based on the state of the specific subject. (Item 3) The imaging device described in item 2, characterized in that the control means controls the imaging device to record the images held for the predetermined time when it is determined that the state of the specific subject satisfies the recording end condition before it is determined that the recording start condition is satisfied. (Item 4) The imaging device described in item 2 or 3, characterized in that when it is determined that a predetermined recording condition is satisfied before it is determined that the state of the specific subject is satisfied, the control means controls the imaging device to start recording images including the images for the predetermined time period that have been retained. (Item 5) The imaging device according to any one of items 2 to 4, characterized in that, after starting recording of the image based on the state of the specific subject, if a predetermined time has elapsed without it being determined that the state of the specific subject satisfies the recording end condition, the control means controls the imaging device to end recording of the image. (Item 6) the recording start condition and the recording end condition are determined for each scene, 6. The imaging device according to any one of items 1 to 5, wherein the control means uses the recording start condition and the recording end condition according to a shooting scene. (Item 7) 7. The imaging device according to any one of items 1 to 6, wherein at least one of the recording start condition and the recording end condition is a condition related to at least one of the posture of the subject, the positional relationship between the subject and surrounding objects, the movement direction of the subject, the movement trajectory of the subject, and depth information. (Item 8) 8. The imaging device according to item 7, wherein at least one of the recording start condition and the recording end condition includes a condition regarding a positional relationship between a subject and a surrounding object. (Item 9) 9. The imaging device according to any one of items 1 to 8, wherein at least one of the recording start condition and the recording end condition is a condition for detecting a motion of a subject. (Item 10) 10. The imaging device according to item 9, wherein the motion of the subject is a motion related to a specific sport. (Item 11) The imaging device is capable of changing the imaging direction, the control means controls the photographing direction so as to track the specific subject. 11. The imaging device according to any one of items 1 to 10, (Item 12) The imaging device is capable of changing an angle of view, the control means controls the angle of view so that an area of ​​the specific subject in the image is equal to or larger than a predetermined size. 12. The imaging device according to any one of items 1 to 11, (Item 13) A control method executed by an imaging device, comprising: Detecting a specific subject from an image obtained using the imaging element; controlling the imaging device to start recording the image when a state of the specific subject satisfies a predetermined recording start condition; controlling the imaging device to end recording of the image when a state of the specific subject satisfies a predetermined recording end condition; 13. A method for controlling an imaging apparatus comprising: (Item 14) 13. A program for causing a computer included in an imaging apparatus to function as each of the means included in the imaging apparatus according to any one of items 1 to 12. [Explanation of symbols]

[0136] 101: imaging device, 104: tilt rotation unit, 105: pan rotation unit, 206: imaging section, 207: image processing circuit, 221: control circuit, 301: external device

Claims

1. A detection means for detecting a subject based on an image obtained using an image sensor, Distance information acquisition means for acquiring information about the distance to the subject, When it is determined that the distance to the subject meets a predetermined first condition, the recording of the image is started. A control means for controlling the imaging device to terminate the recording of the image when it is determined that the distance meets a predetermined second condition, An imaging device characterized by having the following features.

2. The imaging device according to claim 1, characterized in that the distance information is based on depth information.

3. The detection means detects a predetermined type of subject from among the types of subjects that the detection means can detect, The control means starts recording an image when it is determined that the distance to the predetermined type of subject meets the first condition. The imaging device according to claim 1, characterized in that the imaging device is controlled to terminate the recording of the image when it is determined that the distance meets the second condition.

4. The imaging device according to claim 1, characterized in that the control means terminates the recording of the image when a predetermined time has elapsed from the start of recording the image.

5. The imaging apparatus according to claim 1, characterized in that the first condition and the second condition include a condition relating to the distance between the subject and surrounding objects detected in the image.

6. A detection means for detecting the type of subject based on an image obtained using an image sensor, Based on the detection results of the detection means, if it is determined that a predetermined type of subject has been detected, the recording of the image is started. A control means controls the imaging device to terminate recording of the image when it is determined, based on the detection result of the detection means, that no predetermined type of subject has been detected. An imaging device characterized by having the following features.

7. The imaging device according to claim 6, characterized in that the type of subject includes at least one of a person, an animal, or a vehicle.

8. A detection means for detecting a specific subject from an image obtained using an image sensor, Control means for controlling the imaging device such that recording of the image begins when a specific parameter among the parameters relating to the specific subject meets a first condition, and recording of the image ends when a specific parameter among the parameters relating to the specific subject meets a second condition. An imaging device characterized by having the following features.

9. The imaging apparatus according to claim 8, characterized in that at least one of the first condition and the second condition is a condition relating to at least one of the posture of the subject, the positional relationship between the subject and surrounding objects, the direction of movement of the subject, the trajectory of movement of the subject, and depth information.

10. The imaging apparatus according to claim 9, characterized in that at least one of the first condition and the second condition includes a condition relating to the positional relationship between the subject and surrounding objects.

11. The imaging apparatus according to claim 8, characterized in that the control means controls the imaging apparatus to continue holding the image for a predetermined period of time before starting to record the image based on the state of the specific subject.

12. The imaging apparatus according to claim 11, characterized in that the control means controls the imaging apparatus to record the image for the predetermined time being held if it is determined that the state of the specific subject meets the second condition before it is determined that it meets the first condition.

13. The imaging apparatus according to claim 11, characterized in that the control means controls the imaging apparatus to start recording images, including the images held for the predetermined time, if it is determined that the state of the specific subject meets a predetermined third condition before it is determined that it meets the first condition.

14. The imaging apparatus according to claim 11, characterized in that the control means controls the imaging apparatus to terminate the recording of the image if, after starting the recording of the image based on the state of the specific subject, a predetermined time has elapsed without it being determined that the state of the specific subject meets the second condition.

15. The first and second conditions described above are determined for each scene. The imaging apparatus according to any one of claims 8 to 14, characterized in that the control means uses the first condition and the second condition according to the shooting scene.

16. The imaging apparatus according to any one of claims 8 to 14, characterized in that at least one of the first condition and the second condition is a condition for detecting the movement of a subject.

17. The imaging device according to claim 16, characterized in that the action of the subject is an action related to a specific sport.

18. The imaging device is capable of changing the shooting direction. The control means controls the shooting direction to track the specific subject. The imaging apparatus according to any one of claims 8 to 14.

19. The imaging device is capable of changing the field of view. The control means controls the field of view so that the area of ​​the specific subject in the image is larger than a predetermined size. The imaging apparatus according to any one of claims 8 to 14.

20. A control method performed by an imaging device, Detecting a subject from an image obtained using an image sensor and To obtain information regarding the distance to the aforementioned subject, The imaging device is controlled to start recording an image when it is determined that the distance to the subject meets a predetermined first condition. The imaging device is controlled to terminate the recording of the image when it is determined that the distance meets a predetermined second condition. A control method for an imaging device, characterized by having the following features.

21. A control method performed by an imaging device, Detecting the type of subject from an image obtained using an image sensor, Based on the detection results of the subject type, the imaging device is controlled to start when it is determined that a predetermined type of subject has been detected. A method for controlling an imaging device, characterized by controlling the imaging device to terminate the recording of the image when it is determined that, based on the detection result, a predetermined type of subject has not been detected.

22. A control method performed by an imaging device, Detecting a specific subject from an image obtained using an image sensor, Controlling the imaging device to start recording the image when the state of the specific subject meets predetermined recording start conditions, The imaging device is controlled to terminate recording of the image when the state of the specific subject meets a predetermined recording termination condition. A control method for an imaging device, characterized by having the following features.

23. A program for causing a computer in an imaging device to execute each step of the control method for the imaging device described in any one of claims 20 to 22.

Images