Angle-of-view control apparatus, imaging apparatus, imaging system, control method, and program

Abstract

To achieve appropriate framing in accordance with a subject.SOLUTION: An angle-of-view control apparatus for determining settings for angle of view to be applied to an imaging apparatus includes: determination means which determines settings for angle of view to be applied to the imaging apparatus; output means which configures control information related to modification of the settings for angle of view, based on the settings for angle of view determined by the determination means, and output the information to the imaging apparatus; acquisition means which acquires an image captured by the imaging apparatus; and storage means which stores, regarding predetermined settings for angle of view, a template in which a distribution of images of predetermined objects in the angle of view is defined. The determination means is configured to determine the settings for angle of view to be applied to the imaging apparatus on the basis of a difference between the distribution of the predetermined objects in a first captured image captured by the acquisition means for the predetermined settings for angle of view and a distribution of the predetermined objects in the template stored in the storage means for the settings for angle of view.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present invention relates to a field of view control device, an imaging device, an imaging system, a control method, and a program, and more particularly to a technique for remotely adjusting the field of view setting of an imaging device. [Background technology]

[0002] There is an imaging technology that remotely controls pan, tilt, and zoom (PTZ) of an imaging device placed at a predetermined position to obtain an image with a desired angle of view. As a method for automating the setting of the angle of view of such an imaging device, Patent Document 1 discloses a method for adjusting the setting of the angle of view so as to include all of the faces of people detected in the imaging range and an area of ​​interest included in the background, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-035634 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology described in Patent Document 1, if an unintended person is included in the imaging range, the angle of view setting is adjusted to include that person, which could result in a poorly captured image. For example, when capturing images at a wedding ceremony venue, the subjects that should be included in the frame are primarily the bride and groom, but attendants, invited guests, etc. may also be included in the imaging range, so with the technology described in Patent Document 1, there is a possibility that an angle of view setting that will result in a good image will not be selected.

[0005] In contrast, by setting the imaging device to a fixed angle of view without PTZ operation, it is possible to capture images that always capture a specific area of ​​the venue. However, differences in the height, physique, and standing position of the subjects, such as the bride and groom, can sometimes prevent the overall composition from being ideal. In such cases, the operator in charge of photography must manually adjust the imaging device's angle of view setting while viewing the captured image, which is cumbersome.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a field angle control device, an imaging device, an imaging system, a control method, and a program that realize suitable framing according to a subject. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the field of view control device of the present invention is a field of view control device that determines a field of view setting to be applied to an imaging device, and includes: a determination means for determining a field of view setting to be applied to the imaging device; an output means for constructing control information related to a change in the field of view setting based on the field of view setting determined by the determination means and outputting the control information to the imaging device; an acquisition means for acquiring an image captured by the imaging device; and a storage means for storing a template that defines the distribution of images of predetermined objects in the field of view for a predetermined field of view setting, wherein the determination means determines the field of view setting to be applied to the imaging device based on the difference between the distribution of the predetermined objects in a first captured image acquired by the acquisition means for the predetermined field of view setting and the distribution of the predetermined objects in the template stored in the storage means for that field of view setting. [Effects of the Invention]

[0008] With this configuration, the present invention makes it possible to achieve suitable framing according to the subject. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating the configuration of an imaging system according to an embodiment and a modification of the present invention; [Figure 2] FIG. 1 shows an example of the placement of a remote camera 110 according to an embodiment of the present invention. [Figure 3] FIG. 1 illustrates an example of a hardware configuration of a remote camera 110 according to an embodiment and a modification of the present invention. [Figure 4] FIG. 1 is a diagram illustrating a functional configuration of a remote camera 110 according to an embodiment and a modification of the present invention. [Figure 5] FIG. 1 illustrates an example of a hardware configuration of a controller 120 according to an embodiment and a modification of the present invention. [Figure 6] FIG. 10 is a diagram illustrating the functional configuration of a controller 120 according to an embodiment and a modification of the present invention. [Figure 7] FIG. 1 illustrates an example of a hardware configuration of a PC 170 according to an embodiment and a modification of the present invention. [Figure 8] FIG. 10 is a diagram illustrating the functional configuration of a PC 170 according to an embodiment and a modification of the present invention. [Figure 9] FIG. 10 is a diagram illustrating a GUI used to register preset information according to an embodiment of the present invention. [Figure 10] FIG. 10 is a diagram illustrating a GUI for accepting a preset registration number according to an embodiment of the present invention. [Figure 11] FIG. 10 is a diagram illustrating a GUI used to define a template according to an embodiment of the present invention. [Figure 12] 10 is a flowchart illustrating a registration process executed by the PC 170 according to an embodiment of the present invention. [Figure 13] FIG. 10 is a diagram illustrating one aspect of determining a second angle of view setting according to an embodiment of the present invention. [Figure 14] FIG. 10 is a diagram illustrating an example of a view angle control process executed in a remote camera 110 according to an embodiment of the present invention. [Figure 15] FIG. 10 is a diagram illustrating a GUI used to register additional preset information according to a second modified example of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Embodiment] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] In the embodiment described below, an example of an imaging system will be described in which the present invention is applied to a system including a remote camera having a function for controlling a change in the angle of view setting of the imaging device and a controller that issues an instruction to change the angle of view setting of the remote camera. However, the present invention can be applied to any device that can determine the angle of view setting to be applied to the imaging device.

[0012] <<Configuration of Imaging System>> 1 shows the device configuration of an imaging system according to this embodiment. The imaging system according to this embodiment is made up of a plurality of remote cameras 110, a controller 120, a switcher 140, a video storage 150, a PC 170, and a PoE hub 180.

[0013] Each remote camera 110 is an imaging device that captures subject light via a lens 306 and outputs video information. The remote camera 110 has a mechanism for changing the angle of view, such as panning, tilting, and zooming, and performs operations related to changing the angle of view based on control signals received from a controller 120 connected via an RS-422 serial communication cable 130. The video information output by the remote camera 110 is transmitted to the switcher 140 via an SDI cable 160. In the following description, to facilitate understanding of the invention, the video information transmitted from the remote camera 110 to the switcher 140 is described as information consisting of a series of captured images acquired intermittently, but may also include audio information.

[0014] The switcher 140 displays the video information acquired from the multiple remote cameras 110 and selects the video information to be recorded. Connected to the switcher 140 are a display 142 that displays a list of the multiple pieces of acquired video information and a display 144 that displays one piece of video information selected by a selection operation or the like. In the imaging system of this embodiment, the switcher 140 is used to select the video information to be stored (recorded) in the video storage 150, and outputs a video signal of one selected piece of received video information to the video storage 150 for recording. In other words, the video information displayed on the display 144 is transmitted to the video storage 150 for recording.

[0015] The controller 120 accepts operational inputs related to changing the angle of view setting of each remote camera 110, and sends control signals related to the change in angle of view setting to the remote camera 110. The controller 120 is provided with an LCD 126, which displays the current state and setting information of the controller 120. The controller 120 is also provided with a seesaw switch 127 for controlling the zoom state of the remote camera 110 to be controlled, and a joystick 128 for controlling the pan / tilt state. The controller 120 is also provided with buttons 129 for accepting operational inputs related to changing to a preset angle of view setting or calling up functions related to angle of view setting.

[0016] The PC 170 mainly receives operational inputs relating to the settings of the controller 120 and the remote camera 110, and composes information necessary to reflect the settings and transmits the information to each device.

[0017] The PC 170, remote camera 110, controller 120, and switcher 140 are connected by a UTP cable 190 (or a LAN cable) that passes through a PoE hub 180. The remote camera 110 receives power supply from the PoE hub 180 via the UTP cable 190 and is driven.

[0018] For example, in a mode in which an imaging system is used to record video of a church-style wedding ceremony, the remote cameras 110 may be placed on both sides of the altar (110a and 110b) and at the rear of the church (ceremony hall) (110c) as shown in FIG. 2. The operator selects one of the remote cameras 110a to 110c to be controlled using the controller 120, and can change the angle of view setting (angle of view adjustment) of the corresponding remote camera 110 by operating the seesaw switch 127 or the joystick 128. The video information obtained by capturing video with each remote camera 110 is transmitted to the switcher 140 via the PoE hub 180 and displayed on the display 142 of the switcher 140. When the operator selects one piece of video information (or one remote camera 110) using the switcher 140, the video information is displayed on the display 144 and is also recorded in the video storage 150.

[0019] <Hardware Configuration of Remote Camera 110> Next, the hardware configuration of the remote camera 110 will be described with reference to FIG.

[0020] As shown in the figure, the remote camera 110 has a CPU 301 that performs overall control of the remote camera 110 and various calculations. The CPU 301 reads out an OS and programs stored in a ROM 302, expands them in a RAM 303, and executes them to control the remote camera 110. Here, the ROM 302 is a non-volatile memory, and the RAM 303 is a random access memory that is a volatile memory.

[0021] The photoelectric conversion sensor 307 is an imaging element that converts a light beam incident via the lens 306 into an electrical signal (analog image signal) and outputs the signal. The output analog image signal is subjected to development processing and image processing in the image processing device 308, and is converted into a digital image signal or video signal. The image processing device 308 also performs various compression processes and the like to generate video information (video data) in a predetermined encoding format. The device I / O 310 is an interface related to various devices connected to the remote camera 110, and is connected to, for example, a gyro for performing image stabilization processing.

[0022] The network controller 305 controls the transmission and reception of information to and from external devices. The network controller 305 can control the transmission and reception of commands to and from external devices, and can transmit video and audio information. Compressed video in formats such as JPEG, H.264, HEVC, and VVC generated by the image processing device 308 in the remote camera 110 can be transmitted to a network via a UTP cable via the network controller 305. Furthermore, in the imaging system of this embodiment, uncompressed video can also be output from the remote camera 110 to the switcher 140 via the SDI 312.

[0023] The remote camera 110 also includes a motor 309 for controlling the various states of pan, tilt, and zoom related to the angle of view setting, and the state of the aperture. When the motor 309 receives a control signal from the controller 120 via the RS-422 interface 304, it performs driving related to pan, tilt, and zoom based on the control signal.

[0024] These hardware components are connected by a bus 311 and are configured to be able to communicate information with each other.

[0025] <Functional Configuration of Remote Camera 110> The functional configuration of each remote camera 110 will be further described with reference to the block diagram of FIG.

[0026] The control unit 401 controls the operation of each block of the remote camera 110. The control unit 401 realizes its control function by the CPU 301 reading out a program stored in the ROM 302, expanding the program in the RAM 303, and executing the program.

[0027] The storage unit 402 stores information (preset information) about presets of angle of view settings registered in the remote camera 110. The storage unit 402 may be configured, for example, with the ROM 302. Details will be described later, but in one aspect, the preset information includes information about the pan, tilt, and zoom angle of view settings of the remote camera 110. In one aspect, the preset information may include, for example, numerical values ​​such as the angle and magnification of each adjustment item from a reference state, or numerical values ​​of an encoder provided on the motor 309 for each adjustment item.

[0028] The imaging unit 403 captures an image of a subject in an imaging range and outputs the captured image (or a group of captured images, video information) obtained. The imaging unit 403 is realized by the image processing device 308 developing the analog image signal output by the photoelectric conversion sensor 307.

[0029] The detection unit 404 detects an image of a predetermined object included in the imaging range based on the captured image output by the imaging unit 403. In the imaging system of this embodiment, the predetermined object to be detected by the detection unit 404 is a human body, and the detection unit 404 is capable of detecting a position or area in the captured image where an image of a human body appears. Any known method may be adopted as the algorithm for detecting the image of a human body, and for example, a method may be used to identify an area in the image corresponding to a person's face by pattern matching, and detect an area of ​​a specific shape as an image of a human body based on that area. The detection unit 404 is realized by the image processing device 308 analyzing the captured image.

[0030] The determination unit 405 determines the angle of view setting to be adopted by the remote camera 110. As will be described in detail later, in the imaging system of this embodiment, when a template is defined that specifies the distribution of object images for an angle of view setting for which preset information is determined, the angle of view setting is adjusted based on the template. Therefore, when an angle of view adjustment based on the template is necessary, the determination unit 405 determines the angle of view setting after the adjustment. In other words, the angle of view setting performed by the determination unit 405 is not determined based on a control signal from the controller 120 or the like, but is determined in the remote camera 110 based on the current angle of view setting and the template. The determination unit 405 may be a functional configuration realized by the CPU 301 executing a predetermined process.

[0031] The angle-of-view control unit 406 controls operations related to changing the angle-of-view setting of the remote camera 110. In the remote camera 110 of this embodiment, the angle-of-view setting is changed by changing the pan, tilt, and zoom states of the remote camera 110. The angle-of-view control unit 406 controls changes to the angle-of-view setting based on a control signal received from an external device (the controller 120 or the PC 170) and changes to the angle-of-view setting determined by the determination unit 405. More specifically, the angle-of-view control unit 406 derives a drive amount for the drive unit 407 (described below) based on the difference between the current angle-of-view setting (pan value, tilt value, zoom value) and the angle-of-view setting based on the received control signal or the angle-of-view setting determined by the determination unit 405. The angle-of-view control unit 406 then generates control information for causing a drive operation corresponding to the derived drive amount and outputs the control information to the drive unit 407. The angle-of-view control unit 406 may be a functional configuration realized by the CPU 301 executing a predetermined process.

[0032] The driving unit 407 performs driving related to setting the angle of view of the remote camera 110. The driving unit 407 is a motor 309 provided for changing each of the pan, tilt, and zoom. Based on control information received from the angle of view control unit 406, the driving unit 407 executes driving to change at least one of the pan, tilt, and zoom state of the remote camera 110.

[0033] The communication unit 408 realizes a communication function between the remote camera 110 and an external device. The communication unit 408 receives control signals and transmits information indicating the current state of the angle of view setting to the controller 120 via the RS-422 interface 304. The communication unit 408 also transmits uncompressed video to the switcher 140 via the SDI 312 and compressed video to the PC 170 via the network controller 305. The communication unit 408 also receives preset information configured by the PC 170.

[0034] <Hardware configuration of controller 120> Next, the hardware configuration of the controller 120 will be described with reference to FIG.

[0035] As shown in the figure, the controller 120 has a CPU 501 that performs overall control of the controller 120 and various calculations. The CPU 501 reads out an OS and programs stored in a ROM 502, expands them in a RAM 503, and executes them to control the controller 120. Here, the ROM 502 is a non-volatile memory, and the RAM 503 is a random access memory that is a volatile memory.

[0036] The RS-422 interface 504 is an interface for transmitting control signals (or control commands) relating to changes in the angle of view setting to the remote camera 110, which is the control target of the controller 120. The network controller 505 also controls the transmission and reception of information with external devices. The transmission and reception of control signals may also be performed via the network controller 505.

[0037] The controller 120 also includes an LCD 126 that displays information related to the settings of the controller 120, a user interface (UI) that accepts various operational inputs, and an I / O 510 that is an interface for realizing other external information communication. The UI includes a dial 507 that accepts operational inputs related to focus control and aperture control of the remote camera 110. The UI also includes a seesaw switch 127 and a joystick 128 that accept operational inputs related to changing the angle of view setting of the remote camera 110, and a button 129 that accepts operational inputs such as calling up presets related to angle of view settings and switching white balance.

[0038] These hardware components are connected by a bus 511 and are configured to be able to communicate information with each other.

[0039] <Functional Configuration of Controller 120> The functional configuration of the controller 120 will be further described with reference to the block diagram of FIG.

[0040] The control unit 601 controls the operation of each block of the controller 120. The control unit 601 realizes its control function by the CPU 501 reading out a program stored in the ROM 502, expanding the program in the RAM 503, and executing the program.

[0041] The management unit 602 manages the setting status of preset information for each remote camera 110. In this embodiment, the preset information registered for each remote camera 110 is stored in the storage unit 402 of that remote camera 110, so the controller 120 does not need to retain information indicating its specific content. As will be described in detail later, preset information is configured to be identified by a preset registration number, so information regarding which preset information associated with which preset registration number has been registered for each remote camera 110 is shared with the controller 120. The preset registration number may correspond to, for example, a number assigned to a button 129 of the controller 120. An operator can change the angle of view setting associated with the preset information by selecting a desired remote camera 110 and operating the button 129. Therefore, the management unit 602 manages the registration status of preset information for each remote camera 110 as information for determining whether the operating input to the button 129 is valid. Note that the management unit 602 may have a functional configuration implemented by, for example, the ROM 502.

[0042] The status acquisition unit 603 acquires information about the current angle of view setting (for example, pan value, tilt value, and zoom value) for the remote camera 110 to be controlled via the communication unit 605 described below. The status acquisition unit 603 may have a functional configuration that is realized when the CPU 501 detects via the RS-422 interface 504 that information indicating the current pan, tilt, and zoom states has been received from each remote camera 110.

[0043] The operation input unit 604 detects operation inputs made to various user interfaces and sends a control signal corresponding to the operation input to the control unit 601. The operation input unit 605 is, for example, the dial 507, the seesaw switch 127, the joystick 128, and the button 129.

[0044] The communication unit 605 realizes a function for controlling communication between the controller 120 and external devices. The communication unit 605 transmits control signals related to changes in the angle of view setting and receives information on the current angle of view setting to the remote camera 110 via the RS-422 interface 504. The communication control unit 606 also transmits and receives various information to and from the PC 170 via the network controller 505.

[0045] <PC170 hardware configuration> Next, the hardware configuration of the PC 170 will be described with reference to FIG.

[0046] As shown in the figure, the PC 170 has a CPU 701 that performs overall control of the PC 170 and various calculations. The CPU 701 reads out an OS and programs stored in a ROM 702, expands them in a RAM 703, and executes them to control the PC 170. Here, the ROM 702 is a non-volatile memory, and the RAM 703 is a random access memory that is a volatile memory.

[0047] The external storage controller 704 connects to other removable storage devices such as HDDs and controls the reading and writing of data therefrom. The network controller 705 controls the sending and receiving of information to and from external devices. The GPU 706 is a display control device included in the PC 170, and performs various controls related to the configuration and display of the display screen, including the drawing of graphics to be displayed on the display 707. The PC 170 also includes a keyboard 708 and a mouse 709 as UIs for receiving various operational inputs.

[0048] The PC 170 also includes an I / O 710, which is an expansion interface typified by USB, etc. By connecting a converter 712 that converts to RS-422, which is an interface for controlling the remote camera 110, to the I / O 710, it is also possible to control the remote camera 110 from the PC 170.

[0049] These hardware components are connected by a bus 711 and are configured to be able to communicate information with each other.

[0050] <PC170 functional configuration> The functional configuration of the PC 170 will be further described with reference to the block diagram of FIG.

[0051] The control unit 801 controls the operation of each block of the PC 170. The control unit 801 realizes its control function by the CPU 701 reading out a program stored in the ROM 702, expanding it in the RAM 703, and executing it.

[0052] The storage unit 802 stores preset information related to the angle of view settings of each remote camera 110, and, if present, a template that specifies the distribution of objects defined for the preset angle of view settings. The storage unit 802 may be configured, for example, with the ROM 702 or an HDD connected via the external storage controller 704. As will be described in detail later, the template may specify the distribution of main subject images, such as a bride and groom, in an image captured by the remote camera 110 with the angle of view setting determined by the preset information, i.e., the position of these images within the angle of view.

[0053] The image acquisition unit 803 acquires, via a communication unit 806 (described later), video information output by the remote camera 110 for which preset information is to be registered, i.e., a series of captured images. The image acquisition unit 803 may acquire captured images in any manner. For example, the image acquisition unit 803 may receive captured images output by all of the remote cameras 110 in sequence. Alternatively, the image acquisition unit 803 may acquire captured images output by one remote camera 110 selected by the switcher 140 in sequence. The captured images may be, for example, JPEG-compressed video or compressed video in formats such as H.264, HEVC, or VVC. The image acquisition unit 803 may have a functional configuration that is realized when the CPU 701 detects, via the network controller 705, that captured images (video information) have been received from each remote camera 110.

[0054] The registration unit 804 executes various processes related to the registration of preset information for each remote camera 110. As will be described in detail later, the registration unit 804 configures preset information based on an operation input related to a preset angle of view setting, and transmits the preset information to the target remote camera 110 via a communication unit 806 described later to store the preset information. If necessary, the registration unit 804 also configures a template for the angle of view setting to be the preset, defining the distribution of object images in a captured image obtained with the angle of view setting, and transmits the template to the remote camera 110. When the registration unit 804 configures the preset information, it also transmits information indicating that the preset information has been registered in the remote camera 110 to the controller 120 via the communication unit 806. The registration unit 804 may have a functional configuration that is realized by the CPU 701 executing the corresponding process.

[0055] The operation input unit 805 detects operation inputs made to various user interfaces, and sends a control signal corresponding to the operation input to the control unit 801. The operation input unit 805 is, for example, the keyboard 708 and the mouse 709.

[0056] The communication unit 806 realizes a communication control function between the PC 170 and external devices. The communication unit 806 transmits and receives information to and from the remote camera 110 or the controller 120 via the network controller 705. For example, the communication unit 806 receives video information from the remote camera 110 via the network controller 705. Also, for example, when preset information is configured by the registration unit 804, the communication unit 806 transmits information corresponding to the remote camera 110 or the controller 120 via the network controller 705. The communication unit 806 also transmits a control signal related to a change in the angle of view setting of the remote camera 110 to and from the remote camera 110 via the I / O 710 and the converter 712.

[0057] <<Preset and template functions for angle of view settings>> The imaging system of this embodiment is provided with a function that enables remote adjustment of at least the angle of view setting of each remote camera 110 so that video information with a desired framing is acquired by each remote camera 110 for video recording performed using the system. The operator can adjust the angle of view setting of the corresponding remote camera 110 by operating various UIs provided in the controller 120, for example.

[0058] On the other hand, in a case where the placement position of the remote camera 110 and the framing suitable for video recording are predetermined, such as in the case of video recording of a wedding ceremony, it is time-consuming to configure the remote camera 110 so that an operator's operation is required each time to set the angle of view. For this reason, the imaging system of this embodiment is configured to allow a fixed angle of view setting to be pre-registered as preset information for each remote camera 110, as described above. More specifically, in the imaging system, a registrant such as an operator sets the pan, tilt, and zoom states of the remote camera 110 to desired states and then registers the angle of view setting for that state on the PC 170, thereby creating preset information. The created preset information is transmitted from the PC 170 to the remote camera 110, and is used to control the operation of the remote camera 110 when an instruction to change the angle of view setting related to the preset information is received. Furthermore, the fact that the preset information has been registered is shared with the controller 120, and the controller 120 can receive an operation input for changing the angle of view setting related to the preset information, for example, by pressing a predetermined button 129. Therefore, in response to the detection of an operation input to the button 129, the controller 120 transmits a control signal including information that uniquely identifies the preset information to the corresponding remote camera 110. Then, the operation of the remote camera 110 is controlled based on the control signal, thereby realizing a change in the angle of view.

[0059] Even if the angle of view setting determined by such preset information is applied to the remote camera 110 to capture an image, whether the distribution of the main subject image in the captured image is suitable may vary depending on the appearance and position of the main subject. More specifically, for example, in the case of recording a video of a wedding ceremony as shown in FIG. 2, if the main subjects are the bride and groom and their positions at the altar differ from those assumed by the person who registered the preset information, the captured image may not be framed properly. Furthermore, if the height or body type of at least one of the bride and groom differs from that assumed when the preset information was registered, the captured image may not be framed properly, for example, because the head of the main subject may be out of frame. In other words, whether or not a captured image with the framing intended by the person who registered the preset information is obtained is determined not only by the angle of view setting of the remote camera 110, but also by the distribution of the subject images within the angle of view. Therefore, in an imaging system provided with preset information related to angle of view setting, after changing the angle of view of the remote camera 110 based on the preset information, the operator had to manually adjust it while viewing the image captured by the remote camera 110.

[0060] To reduce the operator's operational burden, the imaging system of this embodiment has a function that enables the definition of a template that specifies the distribution of images of a predetermined object in a captured image obtained with a corresponding angle of view setting when registering preset information. That is, the imaging system provides the registrant with a function that enables the registrant to further define the distribution of images of a predetermined object that preferably appears in a captured image for the angle of view setting that is the preset information. In other words, if the registrant of the preset information desires to further define a preferred framing for the corresponding angle of view setting, a template corresponding to the framing is associated with and stored in the preset information. Alternatively, the preset information is configured to include a template. In other words, when such preset information is selected for use, it changes the corresponding angle of view setting and determines the distribution of a predetermined object that serves as a basis for adjustment of the captured image obtained with the angle of view setting. On the other hand, if the registrant does not desire to define framing, no template is associated with the preset information, and only the angle of view setting of the remote camera 110 is changed.

[0061] <GUI for information registration> A graphic user interface (GUI) for registering preset information and defining templates may be, for example, the form shown in Fig. 9. The GUI 900 shown in Fig. 9 is displayed on the display 707 when a request to register preset information is made on the PC 170, and a corresponding process is triggered by an input on the GUI 900 using, for example, the mouse 709.

[0062] As shown in the figure, the GUI 900 includes a captured image 901 of the remote camera 110 (hereinafter referred to as the target camera) that is the registration target, and the registrant can check the framing on the PC 170. The captured image 901 is acquired by the image acquisition unit 803 via the communication unit 806.

[0063] The GUI 900 is also provided with a pan / tilt button 902 and a zoom slider 903, and the registrant can change the pan, tilt, and zoom states of the target camera by operating these buttons. At this time, the control unit 801 sends a control signal corresponding to the operation input to the target camera via the communication unit 806, causing the target camera to perform an operation related to changing the corresponding angle of view setting. In other words, the registrant can determine the angle of view setting to register as preset information for the target camera while checking the imaging state of the target camera in the captured images 901 sequentially sent from the target camera.

[0064] The GUI 900 includes a registration button 904, and in response to an operation input using this button, the angle of view setting currently applied to the target camera is registered as preset information. In the imaging system of this embodiment, each piece of preset information is registered so that it can be identified by a preset registration number. When an operation input is made using the registration button 904, a GUI for accepting the designation of the preset registration number, as shown in FIG. 10, is further displayed. When a preset registration number is input via the GUI, the registration unit 804 composes preset information including the preset registration number. The registration unit 804 then transmits the composed preset information to the communication unit 806, which then transmits the composed preset information together with a registration request to the target camera. At this time, in response to receiving the registration request via the communication unit 408, the control unit 401 of the target camera stores the preset information received together with the registration request in the storage unit 402, thereby completing the registration of the preset information. The registration unit 804 also transmits information indicating that the preset information for the preset registration number has been registered to the controller 120 via the communication unit 806, and causes the management unit 602 to manage this information. The configured preset information is also stored in the storage unit 802.

[0065] When an operation input is made to the cancel button 905 in the GUI 900, the function relating to the registration of the preset information is terminated, and the display of the GUI 900 is also terminated.

[0066] The GUI 900 also has a template registration button 906 for accepting the registration of a template. By operating the template registration button 906, the registrant can define a template related to the angle of view setting currently applied to the target camera and then register the preset information. In other words, when preset information is registered by operating the template registration button 906 instead of the registration button 904, the preset information associated with the template is registered in the remote camera 110.

[0067] When an operation input is made to the template registration button 906, the display on the display 707 switches from GUI 900 to GUI 1100 shown in Fig. 11(a). Like GUI 900, GUI 1100 includes a captured image 1101 of the target camera. By placing an icon corresponding to an object in the captured image 1101, the registrant can specify the distribution of images of a predetermined object that will be used as the adjustment standard within the angle of view related to the captured image 1101.

[0068] Icons that can be placed are listed in an object list 1102, and a registrant can place a desired icon from the object list 1102 on the captured image 1101 for use as a template by dragging and dropping it onto the captured image 1101. In this embodiment, the predetermined object is a human body, and the object list 1102 lists icons corresponding to different human body appearances. In the example shown in the figure, four types of icons are placed to represent the appearance of a human body. Each icon is a full-body front icon 1102a that shows the whole body of a person facing forward, a upper-body front icon 1102b that shows the upper half of the body of a person facing forward, a full-body side icon 1102c that shows the whole body of a person facing to the side, and a face icon 1102d that shows the person's face.

[0069] For example, when an operation input is made to drag and drop a full-body side view icon 1102c from the object list 1102 onto the captured image 1101, a full-body side view icon 1111 is newly placed and displayed on the captured image 1101 as shown in FIG. 11(b). The registrant can determine the initial placement position of the image of the person to be used as a template by this operation input. The full-body side view icon 1111 placed in this manner can be further moved on the captured image 1101, and the position within the angle of view of the corresponding subject image can be changed. Furthermore, by moving each vertex of the outer frame of the full-body side view icon 1111, the size within the angle of view of the corresponding subject image, the posture (tilt), and the horizontal / vertical inversion can be performed.

[0070] The registrant may place (by dragging and dropping or other adjustments) icons corresponding to the desired number of subjects on the captured image 1101, and can specify the distribution of person images (subject images) related to the angle of view setting of the target camera on an icon basis. In other words, the GUI 1100 allows the registrant to specify framing criteria, such as selecting the type of person's appearance and determining where and how large the person's image should be placed in the angle of view.

[0071] It should be noted that the icons are arranged and managed in different upper layers according to their arrangement order, and are configured so that the arrangement can be independently re-edited. By operating the layer adjustment button 1103, the registrant can change the hierarchical relationship of the layers and change the hierarchical relationship between the icons on the captured image 1101, i.e., the front-to-back relationship in the depth direction of the people corresponding to the icons.

[0072] Like GUI 900, GUI 1100 includes a registration button 1104 and a cancel button 1105. When an operation input is made to each of these buttons, the preset information is registered or the function related to the preset registration is terminated. When an operation input is made to the registration button 1104, input of a preset registration number is also accepted. Furthermore, the preset information registered when an operation input is made to the registration button 1104 is associated with a template defined by the distribution of icons arranged on the captured image 1101. More specifically, when a preset registration number is input, the registration unit 804 composes preset information that includes the preset registration number and associates a template configured based on the icons arranged on the captured image 1101. The registration unit 804 then transmits the composed preset information to the communication unit 806, which then transmits the composed preset information together with a registration request to the target camera. At this time, in response to reception of the registration request by the communication unit 408, the control unit 401 of the target camera stores the preset information received together with the registration request in the storage unit 402, thereby completing the registration of the preset information. The registration unit 804 also transmits information indicating that the preset information for the preset registration number has been registered to the controller 120 via the communication unit 806, and causes the management unit 602 to manage the information to that effect. The configured preset information is also stored in the storage unit 802.

[0073] In this way, in the imaging system of this embodiment, preset information that defines the angle of view settings to be applied to each remote camera 110 can be configured and registered in the PC 170, and, if necessary, a template that specifies the distribution of subject images in the captured image related to the angle of view settings.

[0074] In the imaging system of this embodiment, the preset registration numbers correspond to the respective buttons 129 provided on the controller 120, and the buttons 129 with registered preset information can provide a function of changing the state to the angle of view setting of the preset information. That is, by registering preset information associated with any preset registration number in each remote camera 110, the controller 120 can accept an operation input to the button 129 associated with that number. That is, by performing an operation input to the button 129 with registered corresponding preset information, the operator can cause the selected remote camera 110 to perform an operation related to a change to the angle of view setting associated with the preset information.

[0075] At this time, in response to the operation input unit 604 detecting that an operation input has been made to the button 129, the control unit 601 outputs the preset registration number as a control signal to the selected remote camera 110 via the communication unit 605. Based on the control signal, the angle of view control unit 406 of the selected remote camera 110 can read out the corresponding preset information from the storage unit 402 and control the drive of the drive unit 407 so that the angle of view is set as indicated by the preset information.

[0076] <Registration process> Specific registration processing executed by the PC 170 of this embodiment for registering such preset information will be described below with reference to the flowchart in Fig. 12. The processing corresponding to this flowchart can be realized by the CPU 701 reading out a corresponding processing program stored in, for example, the ROM 702, and then loading and executing the program in the RAM 703. This registration processing will be described as being started when, for example, an application related to the imaging system running on the PC 170 detects an operation input related to registering preset information for a target camera. While a detailed description will be omitted below, it is assumed that processing related to displaying the GUI 900 or GUI 1100 is also being executed in parallel during the execution of this registration processing.

[0077] In S1201, the control unit 801 determines whether or not an operational input has been made to change the angle of view setting of the target camera. That is, the control unit 801 determines whether or not an operational input has been made to either the pan / tilt button 902 or the zoom slider 903 of the GUI 900. If the control unit 801 determines that an operational input has been made to change the angle of view setting of the target camera, it proceeds to S1202, and if it determines that an operational input has not been made, it proceeds to S1203.

[0078] In S1202, the control unit 801 transmits a control signal corresponding to the operation input to the target camera via the communication unit 806. As a result, drive control related to the change in the angle of view setting is executed in the target camera, and the angle of view setting of the target camera is changed. At this time, the image acquisition unit 803 can acquire a captured image output by the target camera in the state of the changed angle of view setting.

[0079] In S1203, the control unit 801 determines whether or not an instruction to register preset information for the angle of view setting currently applied to the target camera has been issued. The determination in this step may be made based on whether or not an operation input has been made to the registration button 904. If the control unit 801 determines that an instruction to register preset information has been issued, the control unit 801 proceeds to S1204, and if it determines that an instruction has not been issued, the control unit 801 proceeds to S1206.

[0080] In S1204, the registration unit 804 accepts an input relating to the designation of a preset registration number under the control of the control unit 801. When the registration unit 804 accepts an input relating to the designation of a preset registration number, the processing proceeds to S1205.

[0081] In S1205, the registration unit 804 registers the preset information in the target camera under the control of the control unit 801, and completes this registration process. Specifically, the registration unit 804 acquires information on the pan value, tilt value, and zoom value currently applied to the target camera, and configures preset information together with the preset registration number input in S1204. The registration unit 804 then transmits the configured preset information to the communication unit 806, and sends it to the target camera together with a registration request. The registration unit 804 also transmits information indicating that the preset information has been registered for the preset registration number of the target camera to the controller 120 via the communication unit 806. As a result, the preset information is registered in the target camera and the controller 120, and the target camera and the controller 120 can be made usable (the angle of view setting related to the preset information can be changed).

[0082] On the other hand, if it is determined in S1203 that an instruction to register preset information has not been issued, the control unit 801 determines in S1206 whether or not an instruction to register preset information accompanied by a template definition has been issued. The determination in this step may be made based on whether or not an operation input has been made to the template registration button 906. If the control unit 801 determines that an instruction to register preset information accompanied by a template definition has been issued, the control unit 801 proceeds to S1207, and if it determines that an instruction to register preset information accompanied by a template definition has not been issued, the control unit 801 returns to S1201.

[0083] In S1207, the control unit 801 executes various processes for icon placement related to the definition of the template in the GUI 1100. The processes executed in this step are various information processes related to defining the distribution of subject images in the captured image in response to detection of operation input such as drag-and-drop of an icon from the object list 1102, change of position, change of size, change of attitude, etc.

[0084] In S1208, the control unit 801 determines whether or not an instruction to register preset information accompanied by defining the distribution of the specified subject images as a template has been issued. The determination in this step may be made based on whether or not an operation input has been made to the registration button 1104. If the control unit 801 determines that an instruction to register preset information accompanied by the definition of a template has been issued, the control unit 801 proceeds to S1209, and if it determines that an instruction has not been issued, the control unit 801 returns to S1207.

[0085] In S1209, the registration unit 804 accepts an input relating to the designation of a preset registration number under the control of the control unit 801. When the registration unit 804 accepts an input relating to the designation of a preset registration number, the processing proceeds to S1210.

[0086] In S1210, the registration unit 804, under the control of the control unit 801, registers the preset information in the target camera, completing the registration process. Specifically, the registration unit 804 first creates a template based on the distribution of subject images (icons) in the captured image defined in the GUI 1100. The template may include, for example, information on the type of subject image (full-body front, full-body side, etc.), information on the reference position, rotation angle, size, etc. of the subject image in the captured image, and information defining the anteroposterior relationship with other subject images, based on the state of each arranged icon. Here, the reference position may indicate the coordinates in the captured image of one of the vertices of an outer frame provided to manage the placement of each icon. However, for convenience, in this embodiment, it is assumed to be the center of gravity of the subject image (area) indicated by the icon. Note that the template may simply be configured to define information on the type of subject image, its position in the angle of view, and its size in the angle of view for each subject image included in the template.

[0087] The registration unit 804 also acquires information on the pan, tilt, and zoom values ​​currently applied to the target camera, and configures preset information together with the preset registration number input in S1209 and the configured template. The registration unit 804 then transmits the configured preset information to the communication unit 806 and sends it to the target camera together with a registration request. The registration unit 804 also transmits information indicating that the preset information has been registered for the preset registration number of the target camera to the controller 120 via the communication unit 806. This allows the target camera and controller 120 to be registered with the preset information and made usable (allowing the angle of view setting related to the preset information to be changed). Furthermore, when the preset information is used, the target camera can refer to a template that defines the distribution of subject images that serves as a reference for the angle of view setting related to the preset information.

[0088] In this way, in the imaging system of this embodiment, preset information that defines the preset angle of view setting, and, if necessary, a template that defines the distribution of specified objects in the angle of view setting, can be registered in each remote camera 110.

[0089] <Template-based angle of view control> When the angle of view setting of the remote camera 110 is changed by specifying preset information associated with the template registered in this way (hereinafter referred to as additional preset information), the angle of view control is different from that of normal preset information.

[0090] When the angle of view setting is changed by specifying normal preset information that is not associated with a template, the remote camera 110 is controlled and driven so that the angle of view setting is set to the angle determined by that preset information. In other words, the captured image output by the remote camera 110 after the angle of view setting is changed captures a fixed imaging range, and as described above, depending on the state of the subject, the subject image may not necessarily be framed in a suitable distribution.

[0091] On the other hand, when the angle of view setting is changed by specifying additional preset information, the remote camera 110 is controlled to achieve the angle of view setting. Then, the remote camera 110 is further controlled to obtain a captured image with framing adjusted according to the state of the subject. That is, when the angle of view setting is changed by specifying additional preset information, the change is performed in two stages: first, to the angle of view setting (first angle of view setting) determined by the additional preset information, and then, to the angle of view setting (second angle of view setting) determined by the determination unit 405 according to the state of the subject. For a captured image acquired when the first angle of view setting according to the additional preset information is applied, information on the distribution of the subject (main subject) image assumed at the time of registration can be acquired from the template associated with the additional preset information. Therefore, the determination unit 405 determines the second angle of view setting for the second stage change based on the difference between the distribution of the main subject image in the captured image acquired after applying the first angle of view setting and the distribution of the main subject image in the template.

[0092] The following describes in detail the determination of the second angle of view setting based on the template and the angle of view control related to the change to the second angle of view setting. In the following description, it is assumed that an operation input using additional preset information for the angle of view setting of the target camera is performed in controller 120, and the state of the target camera has transitioned to the first angle of view setting. To facilitate understanding of the invention, the template is described as information that defines the distribution of images of people, who are the main subject, within the angle of view for an area equal in size to the captured image.

[0093] First, the detection unit 404 performs a human body detection process on the captured image output by the imaging unit 403, thereby detecting an area in the captured image where a person's image appears. In this embodiment, since the four types of icons shown in the object list 1102 in Fig. 11 are configured to be arrangable when defining a template, the detection unit 404 detects images of the whole body front, the upper body front, and the whole body side, which are related to the icons indicating the appearance of a human body. At this time, the imaging range related to the first angle of view setting may include other people in addition to the bride and groom who are the main subjects, for example. However, at the stage of the human body detection process in the detection unit 404, no particular determination is made as to what kind of person's image it is, and as many human bodies as possible are detected in the captured image.

[0094] In addition, since the object list 1102 in Figure 11 is configured to allow icons corresponding to human faces to be placed, the detection unit 404 also detects the human face area in the captured image by performing face detection processing on the captured image.

[0095] The detection unit 404 associates with each area detected by the human body detection process and face detection process information indicating which type of human body appearance (full body front, upper body front, full body side, face) the area represents.

[0096] After detecting the areas related to the person appearing in the captured image, the detection unit 404 then performs a matching process to identify, from the detected areas (detection areas), an area corresponding to the main subject defined in the template associated with the additional preset information. In the matching process, a single detection area to be determined is first extracted for each image of the main subject defined in the template. The detection area to be determined for the image of the main subject defined in the template is an area of ​​the same type (full-body front, upper-body front, full-body side, face, etc.) as the image of the main subject, and whose position in the captured image is closest to the reference position of the image of the main subject in the template. Next, in the matching process, the size of the detection area to be determined in the captured image is compared with the size of the image of the main subject in the angle of view of the template, and it is determined whether the difference falls within a predetermined boundary value. The size difference may be evaluated, for example, by the area ratio or the difference in the number of pixels of the detection area to be determined relative to the size of the template. If the result of this determination is that the size difference falls within the boundary value, the detection region to be determined is treated as a region that corresponds to the image of the main subject. Conversely, if the size difference does not fall within the boundary value, the detection region to be determined is treated as a region that does not correspond to the image of the main subject. In the imaging system of this embodiment, the matching process returns a match success when corresponding detection regions have been identified for all images of the main subject defined in the template, and returns a match failure otherwise.

[0097] When the matching process is successful and detection areas in the captured image corresponding to the images of each main subject defined in the template (hereinafter referred to as corresponding subject areas) are identified, determination unit 405 determines a second angle of view setting based on the difference in their distributions. The second angle of view setting is determined so as to reduce the difference between the distribution of the images of the main subject in the template and the distribution of the corresponding subject areas in the captured image.

[0098] In one aspect, as shown in FIG. 13, for example, a second angle of view setting is determined so that a subject captured within a range determined based on a corresponding subject area of ​​a specific main subject in the horizontal direction of the captured image is maintained within the angle of view, and the corresponding subject area is moved to the position of the image of the main subject. FIG. 13(a) shows the distribution of main subject images defined by a template 1300 associated with the additional preset information of the target camera. The template 1300 defines the distribution of two main subject images (groom image 1301 and bride image 1302), and includes information on the positions of their respective centers of gravity. Meanwhile, FIG. 13(b) shows a captured image 1310 obtained when the first angle of view setting associated with the additional preset information is applied to the target camera. As shown in the figure, the captured image 1310 contains the corresponding subject area (groom area 1311) of the groom image 1301 and the corresponding subject area (bride area 1312) of the bride image 1302, both identified by the matching process.

[0099] In this case, the overall impression of the framing of the template is formed primarily by information (captured subjects) from the wider area when the area is divided based on the main subject image. In the example of template 1300, the groom image 1301 and the bride image 1302 are distributed horizontally to the right of the center of the angle of view. Therefore, when the area within the angle of view is divided based on the center of gravity 1303 of the rightmost bride image 1302, the impression of the framing is likely to be formed by the subjects captured in the space 1305 from the center of gravity 1303 to one end of the left side of the angle of view (including end 1304). Therefore, the second angle of view setting is determined so that the subject captured in the space 1315, similarly in the horizontal direction, defined by the center of gravity 1313 of the bride area 1312 in the captured image and the end 1314 at the left end of the captured image, is maintained in the captured image obtained after the angle of view setting is changed.

[0100] For example, if center of gravity position 1303 of bride image 1302 in template 1300 is (x1, y1), the coordinates of end portion 1304 at the same vertical coordinate on the left edge will be (0, y1). Here, suppose that center of gravity position 1313 of bride region 1312 in captured image 1310 is (x1', y1'), and coordinates of end portion 1314 at the same vertical coordinate on the left edge will be (0, y1'). In this case, due to the difference between the coordinates (x1, y1) of center of gravity position 1303 of bride image 1302 and the coordinates (x1', y1') of center of gravity position 1313 of bride region 1312, the captured image may give an impression of being framing different from that assumed when the additional preset information was registered.

[0101] For this reason, as shown in FIG. 13(c), the determination unit 405 determines a second angle of view setting so that, within the changed angle of view, the subject at the end 1314 is similarly captured at one end of the angle of view in the horizontal direction, while the x-coordinate of the center of gravity position 1313 of the bride region 1312 is x1. That is, the second angle of view setting is determined so that the x-coordinate of the bride region 1312 is the same as the x-coordinate of the bride image 1302 defined in the template 1300. In the example of FIG. 13(b), the second angle of view setting is determined so that the center of the angle of view is moved (panned) horizontally to the right, while zooming out to a position where the subject at the end 1314 is captured at the left end of the angle of view. As a result of the angle of view control by the angle of view control unit 406 based on the determined second angle of view setting, the captured image output by the imaging unit 403 will have the appearance shown in FIG. 13(c). That is, the second angle of view setting is determined so that the position (center of gravity position) of the rightmost corresponding subject area in the horizontal direction in the captured image and the position of the subject at the left end of the angle of view are aligned with the reference position of the rightmost main subject image in the horizontal direction in the template and the position at the left end of the angle of view.

[0102] In this embodiment, a method for determining the second angle of view setting based on the distribution of the main subject image in the horizontal direction as shown in FIG. 13 has been described as one aspect, but the present invention is not limited to this. The second angle of view setting may be determined by similarly matching the distribution of the main subject image in the vertical direction to the template, instead of or in addition to the horizontal direction. That is, instead of or in addition to the method for matching the rightmost corresponding subject area and the leftmost subject in the horizontal direction to positions defined in the template, the second angle of view setting may be determined by a method for matching the leftmost corresponding subject area and the rightmost subject in the horizontal direction to the template. Furthermore, a method for matching the topmost corresponding subject area and the bottommost subject in the vertical direction to the template, or a method for matching the bottommost corresponding subject area and the topmost subject in the vertical direction to the template, may also be used to determine the second angle of view setting.

[0103] Furthermore, when determining the second angle of view setting based on the position of the center of gravity of the corresponding subject area and the end of the angle of view that is far from the corresponding subject area, the head area of ​​the main subject may not be included in the angle of view. Therefore, the determination unit 405 may further determine the second angle of view setting that provides suitable framing by taking into consideration the distance between the position of the face area and the end of the angle of view.

[0104] Furthermore, when the main subject is located closer to the target camera / farther away from the target camera, the size of the corresponding subject area may be larger / smaller than the size of the main subject image defined in the template, etc. Therefore, determination unit 405 may further determine the second angle of view setting by changing the zoom so that the size of the corresponding subject area matches the size of the main subject image defined in the template.

[0105] When the determination unit 405 determines the second angle of view setting in this way, the angle of view control unit 406 derives drive amounts for the pan value, tilt value, and zoom value to be driven by the drive unit 407 based on the difference between the determined second angle of view setting and the first angle of view setting. The angle of view control unit 406 then forms control information for causing the drive unit 407 to perform a drive operation corresponding to the derived drive amount, and outputs the control information to the drive unit 407 to drive it. This makes it possible to control the target camera to a state in which the second angle of view setting is applied.

[0106] <Angle of view control processing> 14 will be used to explain in detail the angle-of-view control process executed by the remote camera 110 of this embodiment based on an operation input related to a change in the angle-of-view setting made on the controller 120. The process corresponding to this flowchart can be realized by the CPU 301 reading out a corresponding processing program stored in, for example, the ROM 302, and loading and executing the program in the RAM 303. This angle-of-view control process will be explained as being started when, for example, the communication unit 408 receives a control signal related to a change in the angle-of-view setting from the controller 120.

[0107] In S1401, the control unit 401 determines whether the received control signal (received control signal) includes information (preset registration number) that uniquely identifies the preset information. If the control unit 401 determines that the received control signal includes information that uniquely identifies the preset information, it proceeds to S1402, and if it determines that the received control signal does not include information, it proceeds to S1412.

[0108] In S1402, under the control of the control unit 401, the angle-of-view control unit 406 starts drive control of the drive unit 407 so as to set the angle of view to the angle specified in the preset information (designated preset information) identified by the received control signal. Specifically, the angle-of-view control unit 406 derives a drive amount for the drive unit 407 based on the difference between the current angle of view setting and the angle of view setting specified in the designated preset information. The angle-of-view control unit 406 then forms control information corresponding to the derived drive amount and outputs it to the drive unit 407, thereby starting the operation of changing the angle of view to the angle of view related to the designated preset information.

[0109] In S1403, the angle-of-view control unit 406 determines whether or not the operation related to the change to the angle-of-view setting defined in the specified preset information has been completed under the control of the control unit 401. If the angle-of-view control unit 406 determines that the operation related to the change to the angle-of-view setting has been completed, it proceeds to S1404, and if it determines that the operation has not been completed, it repeats the processing of this step.

[0110] In S1404, the control unit 401 determines whether the designated preset information is preset information associated with a template (additional preset information). Specifically, the control unit 401 refers to the designated preset information stored in the storage unit 402, and determines whether a template is associated with the designated preset information. If the control unit 401 determines that the designated preset information is additional preset information, it proceeds to S1405, and if it determines that the designated preset information is not additional preset information, it ends this angle of view control process.

[0111] In S1405, under the control of the control unit 401, the detection unit 404 executes human body detection processing on the captured image output by the imaging unit 403, and stores the detection result.

[0112] In S1406, under the control of the control unit 401, the detection unit 404 executes face detection processing on the captured image and stores the detection result.

[0113] In S1407, under the control of the control unit 401, the detection unit 404 executes template-based matching processing on the detection area that is the detection result of the human body detection processing in S1405 and the face detection processing in S1406.

[0114] In S1408, the control unit 401 determines whether the matching process has been successful. If the matching process has been successful, the control unit 401 proceeds to S1409, and if the matching process has failed, the control unit 401 completes this registration control process.

[0115] In S1409, under the control of the control unit 401, the determination unit 405 determines the second angle of view setting based on the difference between the distribution of the corresponding subject area obtained as a result of the matching process and the distribution of the main subject image defined in the template.

[0116] In S1410, under the control of the control unit 401, the angle-of-view control unit 406 starts drive control of the drive unit 407 so as to achieve the second angle-of-view setting determined in S1409. Specifically, the angle-of-view control unit 406 derives a drive amount for the drive unit 407 based on the difference between the second angle-of-view setting and the current angle-of-view setting (first angle-of-view setting). The angle-of-view control unit 406 then forms control information corresponding to the derived drive amount and outputs it to the drive unit 407, thereby starting an operation to change to the angle-of-view setting related to the specified preset information.

[0117] In S1411, the angle-of-view control unit 406 determines whether or not the operation related to the change to the second angle-of-view setting has been completed under the control of the control unit 401. If the angle-of-view control unit 406 determines that the operation related to the change to the second angle-of-view setting has been completed, it completes this angle-of-view control process, and if it determines that the operation has not been completed, it repeats the process of this step.

[0118] On the other hand, if it is determined in S1401 that the reception control signal does not include information that uniquely identifies the preset information, the view angle control unit 406 performs drive control of the drive unit 407 based on the reception control signal in S1412 under the control of the control unit 401. When this drive control is completed, the view angle control unit 406 completes this view angle control process.

[0119] As described above, according to the imaging system of this embodiment, when registering a field of view setting to be applied to the imaging device as a preset, it is possible to define a template that specifies the distribution of predetermined objects in the field of view that serves as a framing reference. Furthermore, when a request to change the field of view setting that specifies preset information associated with a template is received, the field of view setting is changed to one that results in framing closer to the template, based on the captured image obtained after changing to the field of view setting associated with the preset information. This allows the imaging system of this embodiment to achieve optimal framing according to the subject.

[0120] This imaging system can achieve stable framing of video recording, particularly in situations where a suitable framing for a specific imaging range is predetermined but the state of the subject within the angle of view may change each time the system is used, such as when recording video of a wedding ceremony. Furthermore, with the imaging system of this embodiment, the angle of view setting of each imaging device is adjusted according to the state of the subject within the angle of view, and the captured image is output, so the operator can easily select the video information to be recorded using switcher 140 without having to make individual adjustments.

[0121] [Variation 1] In the registration process of the above-described embodiment, the registrant has performed an operation input on the pan-tilt button 902 and the zoom slider 903 in the GUI 900 to accept a template definition for the angle of view setting applied to the remote camera 110. However, the present invention is not limited to this, and may be configured to accept additional template definitions for angle of view settings related to preset information that has already been registered.

[0122] [Variation 2] In the above-described embodiment, as shown in Fig. 11, a captured image obtained when a predetermined object is not present in the imaging range is received, and a template is defined for the captured image, but the present invention is not limited to this. Template definition does not need to be performed for captured images received from the target camera in real time, and may be performed by referring to video information captured in the past by the target camera and recorded in video storage 150, for example.

[0123] In particular, in applications such as recording video of a wedding ceremony, where the basic ceremony progression and the angle of view settings and framing scenarios to be used when recording the ceremony are fixed, it is efficient to be able to define templates based on video information captured in the past by the target camera. That is, rather than manually changing the angle of view settings of the target camera and defining templates by arranging icons for each angle of view setting, it is more convenient for the registrant to be able to define templates based on the distribution of main subject images in frames of already recorded video information. That is, this modified embodiment is convenient in that the registrant can define templates based on the subject images in frames based on video information recorded with a preferred angle of view setting change transition, without having to manually adjust the target camera and imagine the distribution of the main subject images.

[0124] Trace memory function In this modification, the imaging system is provided with a trace memory function capable of recording and reading angle-of-view history information indicating the history of operations related to changes in the angle-of-view setting of each remote camera 110. With this function, each time the angle-of-view setting of the remote camera 110 is changed, a history of the angle-of-view setting and the time (or frame) of the change is added in chronological order and recorded as angle-of-view history information. In other words, this function does not add new history during a period when the angle-of-view setting is not changed, i.e., while the same angle-of-view setting is maintained. Therefore, by referencing the continuous history of the angle-of-view history information, it is possible to identify the period during which imaging was performed with the same angle-of-view setting. While the angle-of-view history information may be recorded and managed by any device, in this modification, it is recorded in the video storage 150.

[0125] Furthermore, video information captured by each remote camera 110 is also recorded in the video storage 150. That is, in this modification, video information related to each remote camera 110 and field of view history information, which is a trace memory at the time of recording of the video information, can be acquired from the video storage 150. According to this information, for video information related to one remote camera 110, information on the field of view setting applied to that remote camera 110 for each frame of the video information of that remote camera 110 can be referenced from the field of view history information. In other words, by referencing information related to one history included in the field of view history information related to one remote camera 110, it is possible to identify the frame section captured by that remote camera 110 with the field of view setting of that history set.

[0126] <<GUI for information registration>> Hereinafter, a method for registering additional preset information for the imaging system of this modified example will be described using a GUI different from that of the embodiment described above. The GUI for defining additional preset information used in the imaging system of this modified example may be, for example, the form shown in Fig. 15(a). The GUI 1500 shown in Fig. 15(a) is displayed on the display 707 when a request for registration of additional preset information based on video information is made in the PC 170, and a corresponding process is triggered by input on the GUI 1500 using, for example, the mouse 709.

[0127] The field of view history information of the target camera for which the registration of additional preset information is desired is read out prior to displaying GUI 1500. In the following example, for ease of explanation, it is assumed that the video storage 150 stores one piece of video information output by the target camera while different field of view settings are applied, and the field of view history information of the target camera is associated with that video information.

[0128] As shown in the figure, GUI 1500 includes a video frame 1501 included in the video information related to the target camera, allowing the registrant to check the framing of the frame. Which frame of the video information is displayed in video frame 1501 is controlled by which of the history of angle of view settings included in the angle of view history information is selected. GUI 1500 also includes a next history button 1503 and a previous history button 1504 for switching the history to be selected for the angle of view history information related to the target camera. By operating these buttons, the registrant can specify the history corresponding to the desired angle of view setting. More specifically, operating next history button 1503 selects the history of angle of view settings next to the currently selected history among the history of angle of view settings included in the angle of view history information. On the other hand, operating previous history button 1504 selects the history of angle of view settings previous to the currently selected history among the history of angle of view settings included in the angle of view history information. This allows the registrant to sequentially refer to the contents of frames when the angle of view setting was changed for pre-recorded video information.

[0129] When GUI 1500 for the target camera is displayed, the first history (oldest angle of view setting) among the angle of view history information is selected, and a frame of the video information captured at the same time as the history is first displayed in video frame 1501. That is, the first history among the angle of view history information is selected as the initial value. Here, if the history indicates angle of view settings for multiple frames, that is, if the angle of view setting was maintained over multiple frames, multiple frames in the video information are assigned to the history, and any one of these frames can be displayed in video frame 1501. More specifically, when one history is selected, if there are different frames captured with the same angle of view setting, the registrant can change the frame displayed in video frame 1501 by operating slider 1502. Slider 1502 is a GUI that presents a movable operating member when multiple frames corresponding to one history exist, and identifies one frame in a frame section captured using the same angle of view setting depending on the operating state of the member.

[0130] In the captured image displayed in video frame 1501, predetermined objects are detected in the same manner as in the angle-of-view control process. In the example of FIG. 15(a), in the case of video recording of a wedding ceremony, the types of predetermined objects to be detected are the full-body front, the front upper body, the full-body side, and the face, as described above. Once the frame to be displayed in video frame 1501 is determined and a human body or the like is detected in the captured image of that frame, a frame is added to the corresponding subject image (to be detected) and the image is displayed as a detection area, as shown in the figure. In the example shown, detection area 1506a relating to the groom, detection area 1506b relating to the bride, and detection areas 1506c and d relating to the guests are detected and displayed with dashed frames.

[0131] In GUI 1500, a registrant can register additional preset information by selecting a detection area associated with a subject they wish to define as a template from among detection areas 1506a-d displayed in displayed video frame 1501. For example, if the registrant wishes to define the distribution of subject images of a bride and groom as a template for the angle of view setting associated with displayed video frame 1501, the registrant can select detection areas 1506a and 1506b as the main subject image. The selected detection area 1506 is clearly indicated by a bold frame (selected area 1511), as shown in FIG. 15(b), and is subsequently registered in response to an operation input using next history button 1503, previous history button 1504, end button 1505, or the like. More specifically, the registration unit 804 obtains the angle of view setting associated with the selected history from the angle of view history information to construct preset information, and then constructs a template based on the position and size of selected area 1511 on video frame 1501 and the type of image, and registers the template in association with the selected area. Here, the type of image that appears in the selection area 1511 is determined by the detection method (full body front, upper body front, full body side, face) used to detect the selection area 1511. This provides a function that enables a registrant to select a desired subject image from subject images included in the frame of video information and easily register additional preset information based on the corresponding angle of view history information.

[0132] Note that the definition of a template based on video information is not limited to one that directly reflects the detection results for the frame. For example, as shown in the detection method list 1512 of FIG. 15(b), the detection algorithm used to detect the selected region 1511 may be displayed as an icon, allowing the desired detection method to be changed when defining the template. In this case, the detection method may be changed by selecting another icon arranged in the detection method list 1512. For example, when a detection region is determined for the displayed video frame 1501, instead of indicating the region using a frame, a corresponding icon may be displayed superimposed and edited to allow adjustment of the position and size. Furthermore, a group of additional preset information in time series may be registered upon completion of selection of the selected region 1511 for all of the time series histories shown in the angle of view history information.

[0133] [Variation 3] In the angle of view control process of the above-described embodiment, when the designated preset information is additional preset information, an operation related to changing to the first angle of view setting is first performed, and then a process related to determining the second angle of view setting is executed based on the captured image obtained after the operation is completed. According to this method, a comparison can be made with the distribution of the main subject image defined in the template using a captured image obtained with the angle of view setting under the same conditions as when the preset information was registered, thereby improving the accuracy of the matching process.

[0134] On the other hand, for example, when the distance between remote camera 110 and the main subject is far, even if the pan, tilt, and zoom states are not exactly the same as those of the first angle of view setting, as long as they are close, the distribution of the main subject image appearing in the captured image will show a similar tendency. That is, during the angle of view control process, when the angle of view setting of remote camera 110 approaches the first angle of view setting, the distribution of the main subject image in the captured image output from imaging unit 403 will be similar to the distribution of the main subject image in the captured image output when the angle of view setting is the first angle of view setting. Therefore, the various processes performed by detection unit 404 and determination unit 405 in determining the second angle of view setting can be realized with the same accuracy as those in the above-described embodiment, even if they are executed without transitioning to the first angle of view setting.

[0135] Therefore, the process of S1405 of the angle of view control process may be started, for example, when the current angle of view setting approaches the first angle of view setting, rather than when it is determined in S1403 that the operation related to the change to the first angle of view setting has been completed. That is, the captured images used in the series of processes related to determining the second angle of view setting may be captured images output by the imaging unit 403 after it is detected that the angle of view setting has approached the first angle of view setting after the operation related to the change to the first angle of view setting has begun. That is, the series of processes related to determining the second angle of view setting are controlled so that they are not performed on captured images obtained after the angle of view setting has approached the first angle of view setting, but on captured images obtained after the angle of view setting has approached the first angle of view setting. Here, whether the current angle of view setting is approaching the first angle of view setting may be determined, for example, by whether the angle between the imaging direction defined by the current angle of view setting and the imaging direction defined by the first angle of view setting is smaller than a threshold.

[0136] By doing this, a series of processes related to determining the second angle of view setting can be started without waiting for the completion of the operation related to changing to the first angle of view setting, thereby reducing the time required to execute the angle of view control process compared to the above-mentioned embodiment.

[0137] Note that, depending on the speed of the operation, image blurring may occur in the captured image obtained during the period when the operation related to the change of the angle of view setting is being performed, which may reduce the accuracy of deriving the second angle of view setting that provides suitable framing. For this reason, the operation related to the change to the first angle of view setting may be controlled so that the change is performed at a normal speed until the angle approaches the first angle of view setting, and then slows down after the angle approaches the first angle of view setting. This can improve the clarity of the captured image used in the series of processes related to determining the second angle of view setting, and increase the accuracy of deriving the second angle of view setting.

[0138] [Variation 4] In the above-described embodiment, a mode has been described in which preset information and templates are stored in each remote camera 110, and when a request to change the angle of view setting specifying the preset information is received, the determination unit 405 determines the angle of view setting based on the template. That is, in the above-described embodiment, a mode has been described in which the remote camera 110, which is an imaging device according to the present invention, is provided with the functional configuration of a field of view control device that, when a change to a predetermined field of view setting is received, controls the change to the field of view setting corresponding to the associated template.

[0139] However, the present invention is not limited to this embodiment, and the functional configuration of the angle of view control device (particularly the storage unit 402, the detection unit 404, and the determination unit 405) may be provided in another device, such as the controller 120 or the PC 170, rather than in the remote camera 110. For example, in an aspect in which the PC 170 has the functional configuration, the PC 170 also receives information indicating that an operation input related to a change in the angle of view setting that specifies additional preset information has been made in the controller 120. Then, the PC 170 performs various detection processes and matching processes on the captured image after the first angle of view setting has been applied, and determines the second angle of view setting. Then, control information for changing to the second angle of view setting is transmitted from the PC 170 to the remote camera 110, and drive control is performed in the remote camera 110.

[0140] [Variation 5] In the above-described embodiment, the predetermined object whose distribution is defined in the template is a person, and in the matching process, the area that is closest in position and whose size difference falls within the boundary value is identified as the corresponding subject area from among the detection areas with the same type of human body appearance. However, the implementation of the present invention is not limited to this, and for example, prior to the start of video recording using additional preset information, the face pattern of the main subject (bride and groom) may be registered, and the corresponding subject area may be identified after recognizing the main subject.

[0141] [Variation 6] In the above-described embodiment, the predetermined object whose distribution is defined in the template is described as a person, but the predetermined object that is the target of the registration of preset information and the detection process may also be other types of subjects, such as a car or an animal.

[0142] [Variation 7] In the above-described embodiment, the angle of view control process is executed in response to receiving a control signal related to a change in the angle of view setting from controller 120, but the present invention is not limited to this. In particular, a part of the process that determines the second angle of view setting based on the difference from the distribution of the main subject image defined in the template may be executed by detecting, for example, movement of the main subject during image capture.

[0143] Furthermore, because such dynamic changes in the angle of view setting may change the framing desired by the operator, the remote camera 110 may be controlled not to be performed while the remote camera 110 is selected for video recording. Whether the remote camera 110 is selected for video recording may be determined by referring to the tally state of the remote camera 110, which changes in response to a control signal from the switcher 140. For example, when the remote camera 110 is in the tally state, the angle of view setting may not be changed even if there is a difference from the distribution of the main subject image defined in the template, and the change may be performed only when the remote camera 110 is not in the tally state. Alternatively, whether or not to control the angle of view according to the tally state may be configured to be selectable as an option.

[0144] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0145] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0146] 401: control unit, 402: storage unit, 403: imaging unit, 404: detection unit, 405: determination unit, 406: angle of view control unit, 407: drive unit

Claims

1. A determination means for determining a field of view setting to be applied to an imaging device; an output unit that generates control information related to a change of the angle of view setting based on the angle of view setting determined by the determination unit and outputs the control information to the imaging device; An acquisition means for acquiring an image captured by the imaging device; a storage means for storing a template defining a distribution of images of a predetermined object within the field of view for a predetermined field of view setting; having The determining means determines a view angle setting to be applied to the imaging device based on a difference between a distribution of the predetermined object in the first captured image acquired by the acquiring means for the predetermined view angle setting and a distribution of the predetermined object in the template associated with the view angle setting and stored in the storage means. A field of view control device comprising:

2. The determining means is a detection means for detecting the predetermined object included in the first captured image; an identification means for identifying the predetermined object, the distribution of which is defined in the template related to the predetermined angle of view setting, from among the predetermined objects detected by the detection means; Including, The determining means determines a view angle setting to be applied to the imaging device so as to reduce a difference between a distribution of the predetermined object in the first captured image identified by the identifying means and a distribution of the predetermined object in the template related to the predetermined view angle setting.

2. The field angle control device according to claim 1,

3. 3. The apparatus according to claim 2, wherein the template defines, for each of the predetermined objects, information on the type of the object, its position within the angle of view, and its size within the angle of view.

4. the predetermined object includes a human body; The predetermined object types include a front surface of a human body, a side surface of a human body, and a predetermined part of a human body.

4. The field angle control device according to claim 3,

5. The view angle control device according to claim 4 , wherein the predetermined object type includes a face of a registered person.

6. 6. The viewing angle control device according to claim 3, wherein the predetermined object includes a specific animal.

7. 7. The apparatus according to claim 3, wherein the template defines a distribution of a plurality of objects as a distribution of the predetermined object.

8. 8. The angle-of-view control device according to claim 3, wherein the determining means determines an angle-of-view setting to be applied to the imaging device so that, in at least one of a horizontal direction and a vertical direction within an angle of view, a subject appearing at one end of the first captured image in that direction is captured at one end of the angle of view, and at least one of the specified objects identified by the identifying means is captured at a position of a corresponding object in the template related to the predetermined angle of view setting.

9. The camera further includes an instruction unit for receiving an instruction to change the predetermined angle of view setting, when an instruction to change to the predetermined angle of view setting is accepted by the instruction means, the output means outputs the control information related to the change to the predetermined angle of view setting to the imaging device; The first captured image is a captured image acquired after the angle of view setting of the imaging device reaches a value close to the predetermined angle of view setting based on the control information output by the output means. The field angle control device according to any one of claims 1 to 8.

10. The angle-of-view control device according to claim 9, characterized in that the determination means does not perform processing to determine the angle-of-view setting to be applied to the imaging device for a second captured image acquired before the angle-of-view setting of the imaging device reaches a vicinity of the predetermined angle-of-view setting.

11. the output means configures the control information so as to decelerate an operation related to a change of a field of view setting when the field of view setting of the imaging device approaches the predetermined field of view setting; The first captured image is an captured image acquired after the angle of view setting of the imaging device approaches the predetermined angle of view setting and an operation related to the change of the angle of view setting is decelerated based on the control information output by the output means.

11. The field angle control device according to claim 9, wherein the field angle control device is a field angle control device.

12. The angle-of-view control device according to claim 11, characterized in that the determination means does not perform processing to determine the angle-of-view setting to be applied to the imaging device for a third captured image acquired before an operation related to changing the angle-of-view setting of the imaging device is decelerated.

13. 13. The angle-of-view control device according to claim 1, further comprising a generation unit that generates the template based on an operation input related to designation of a distribution of images of the predetermined object for setting an angle of view of the imaging device, and stores the template in the storage unit.

14. 14. The device according to claim 1, wherein the setting of the angle of view of the imaging device determines at least one of a pan, tilt, and zoom state of the imaging device.

15. a determination unit for determining a tally state of the imaging device, The output means outputs the control information so as to change the angle of view setting when the imaging device is not in a tally state. The field angle control device according to any one of claims 1 to 14.

16. An imaging means; A field angle control device according to any one of claims 1 to 15, a driving means for driving a state of the imaging device to a field angle setting corresponding to the control information output by the field angle control device; An imaging device comprising:

17. An imaging system including an imaging device having a driving means for changing an angle of view setting, and an angle of view control device, The angle of view control device includes: A determination means for determining a view angle setting to be applied to the imaging device; an output unit that generates control information for the driving unit related to a change in the angle of view setting based on the angle of view setting determined by the determination unit, and outputs the control information to the imaging device; An acquisition means for acquiring an image captured by the imaging device; a storage means for storing a template defining a distribution of images of a predetermined object within the field of view for a predetermined field of view setting; having The determining means determines a view angle setting to be applied to the imaging device based on a difference between a distribution of the predetermined object in the first captured image acquired by the acquiring means for the predetermined view angle setting and a distribution of the predetermined object in the template associated with the view angle setting and stored in the storage means.

1. An imaging system comprising:

18. A determination step of determining an angle of view setting to be applied to an imaging device; an output step of configuring control information related to a change of the angle of view setting based on the angle of view setting determined in the determination step, and outputting the control information to the imaging device; an acquisition step of acquiring an image captured by the imaging device; A control method for a field of view control device having the field of view control device includes a storage means for storing a template that defines a distribution of images of a predetermined object within a field of view for a predetermined field of view setting; In the determining step, a view angle setting to be applied to the imaging device is determined based on a difference between a distribution of the predetermined object in the first captured image acquired in the acquiring step for the predetermined view angle setting and a distribution of the predetermined object in the template associated with the view angle setting and stored in the storage means. A control method comprising:

19. A computer comprising: A process of determining an angle of view setting to be applied to the imaging device; A process of forming control information related to a change of a field of view setting based on the field of view setting determined in the process of determining, and outputting the control information to the imaging device; A process of acquiring an image captured by the imaging device; Run the command, the computer includes a storage means for storing a template that defines a distribution of images of a predetermined object within a field of view for a predetermined field of view setting; In the determining process, a view angle setting to be applied to the imaging device is determined based on a difference between a distribution of the predetermined object in the first captured image acquired in the acquiring process for the predetermined view angle setting and a distribution of the predetermined object in the template associated with the view angle setting and stored in the storage means. A program characterized by:

Images