Angle-of-view control apparatus, angle-of-view control method, and

Abstract

To provide a technique capable of reducing a burden on an operator and automatically controlling an angle of view.SOLUTION: An angle-of-view control device is an angle-of-view control device that controls an angle of view of a photographing device, and includes a template search unit that searches for a template applicable to control of the angle of view from a plurality of templates including information regarding arrangement of a predetermined object, and an angle-of-view change unit that changes and controls the angle of view by controlling at least one of pan, tilt, and zoom of the photographing device based on arrangement of the object designated in the searched template.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a field of view control device, a field of view control method, and a program. [Background technology]

[0002] 2. Description of the Related Art Techniques for automatically controlling the angle of view of a camera by adjusting pan, tilt, and zoom are known.

[0003] For example, Patent Document 1 discloses a technology for setting the angle of view based on a template that defines the distribution of object images. In Patent Document 1, a preset is searched for based on a preset registration number entered by an operator, and the template contained in the preset is acquired. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-118466 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the technology of Patent Document 1, the operator must input a preset registration number, in other words, the operator must select a preset that has a template, which places a heavy burden on the operator.

[0006] Therefore, the present invention provides a technique that can reduce the burden on the operator and automatically control the angle of view. [Means for solving the problem]

[0007] In order to solve this problem, for example, the field of view control device of the present invention has the following configuration: A field of view control device for controlling a field of view of an imaging device, a template search means for searching for a template applicable to the control of the angle of view from a plurality of templates each including information on the arrangement of a predetermined object; a view angle changing means for changing and controlling the view angle by controlling at least one of panning, tilting, and zooming of the image capturing device based on the arrangement of the object specified in the searched template; It has. [Effects of the Invention]

[0008] According to the present invention, the angle of view can be automatically controlled while reducing the burden on the operator. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram illustrating the overall configuration of an imaging system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of the camera according to the embodiment. [Figure 3] FIG. 2 is a block diagram showing the software functional configuration of the camera of the present embodiment. [Figure 4] FIG. 10 is a diagram showing an example of a template information table for controlling the angle of view. [Figure 5] 5A to 5C are diagrams illustrating examples of ranges included in template information. [Figure 6] FIG. 4 is a flowchart of a view angle control process executed by the camera of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] (Embodiment) 1 is a schematic diagram illustrating the overall configuration of an imaging system according to this embodiment, and an example of the configuration of the imaging system according to this embodiment will be described with reference to FIG.

[0012] The imaging system of this embodiment includes a camera 101, which is an example of a field angle control device and an imaging device, and an operation input device 103. The camera 101 and the operation input device 103 are connected via a network 102 so as to be able to send and receive data.

[0013] The camera 101 receives light from a subject and converts the light into an electrical signal to generate image (also referred to as a captured image) data. The camera 101 has an image sensor, such as a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) sensor, that converts light into an electrical signal to generate an image. The term "image" may include a captured image, a still image, a video, a video, and data thereof. The camera 101 has pan, tilt, and zoom mechanisms, allowing the angle of view to be changed.

[0014] The camera 101 controls the angle of view in accordance with a control signal input from the operation input device 103 and information detected from a captured image. The camera 101 can switch between a template capture mode, an automatic capture mode, and a manual capture mode. In the automatic capture mode, the camera 101 automatically performs PTZ (pan-tilt-zoom) control based on captured images or information acquired by devices such as sensors to track and capture images of people, animals, drones, and other moving objects (hereinafter, "moving objects"). However, the automatic capture mode may also be a state in which the camera 101 or an information processing device connected to the camera 101 controls the PTZ without receiving instructions from an operator. The camera 101 changes its operation in accordance with data such as a control signal received from the operation input device 103, and is controlled automatically or by the control signal.

[0015] The operation input device 103 may be a controller, a personal computer, a smartphone, a tablet terminal, etc. The operation input device 103 may be a terminal that can transmit a control signal to the camera 101 by manual operation by an operator, etc. The operation input device 103 accepts operations such as input from the operator and transmits various data such as control signals to the camera 101.

[0016] 1 is an example showing the main components of the imaging system of this embodiment, which will be referred to in the following description. Therefore, the imaging system may include additional devices in addition to the components shown in the figure. For example, the imaging system may have multiple cameras connected to the network 102. The imaging system may also have a server that distributes images and a server that has a function of storing images via the network 102.

[0017] 2 is a block diagram showing an example of the hardware configuration of the camera 101. The hardware configuration of the camera 101 will be described with reference to FIG.

[0018] The camera 101 has a CPU 201, RAM 202, storage 203, operation unit 204, output control unit 205, communication I / F 206, and bus 207. The CPU 201, RAM 202, storage 203, operation unit 204, output control unit 205, and communication I / F 206 are connected via the bus 207 so as to be able to transmit and receive data to and from each other. In Fig. 2, the camera 101 and operation input device 103 are connected to a LAN (Local Area Network), but the network 102 connecting the respective devices can take various forms, and the network 102 is not limited to a specific form.

[0019] The CPU 201 stands for Central Processing Unit and is a processor that executes various types of arithmetic processing. Instead of or in addition to the CPU 201, the camera 101 may include one or more other processors, such as an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or a QPU (Quantum Processing Unit). The CPU 201 executes various types of control of the camera 101. For example, the CPU 201 controls motors 208P, 208T, and 208Z connected to the camera 101 for executing PTZ control via the output control unit 205. The CPU 201 acquires data via the operation unit 204. The CPU 201 may also output generated data to an output device via the output control unit 205. The CPU 201 executes various types of control and functions of the camera 101 by reading computer programs from the storage 203 and loading them onto the RAM 202. For example, the CPU 201 loads a computer program and executes a field of view control process, which will be described later. The CPU 201 may acquire and execute the above-described program from another device via the network 102 and the communication I / F 206. When executing the image analysis and distribution program, the CPU 201 stores image data in the RAM 202 and reads the image data during execution of the program. The CPU 201 may store analysis results, such as the detection of people in the image, in the RAM 202.

[0020] The RAM 202 is an abbreviation for Random Access Memory. The RAM 202 is a storage medium that can be read and written at high speed. The RAM 202 functions as a work area when the CPU 201 executes a program.

[0021] The storage 203 stores a boot program executed by the CPU 201 when the camera 101 starts up, an instruction program for executing each process, data such as image data processed by the program, and data such as parameters used by the program. The storage 203 may be a readable nonvolatile storage medium such as a ROM (Read Only Memory), or a readable / writable nonvolatile storage medium such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0022] The operation unit 204 processes signals received from the operation input device 103 via the communication I / F 206. The operation unit 204 outputs the processed signals to the CPU 201.

[0023] The output control unit 205 is connected to output devices such as motors 208P, 208T, and 208Z for PTZ control. The output control unit 205 outputs control signals received from the CPU 201 to the motors 208P, 208T, and 208Z. The motor 208P controls panning. The motor 208T controls tilting. The motor 208Z controls zooming. The motors 208P, 208T, and 208Z may each be one or more motors. In the following description, when there is no need to distinguish between the motors 208P, 208T, and 208Z, they may be referred to as motors 208. Furthermore, instead of or in addition to motors, the camera 101 may have actuators other than motors for PTZ control.

[0024] The communication I / F 206 receives data from other devices such as the operation input device 103 via the network 102 and sends the data to the CPU 201. The communication I / F 206 transmits data generated by the CPU 201 to other devices via the network 102.

[0025] Alternatively, the camera 101 may have hardware such as one or more integrated circuits, such as an application specific integrated circuit (ASIC) and a programmable logic device (PLD) including a field programmable gate array (FPGA).

[0026] Fig. 3 is a block diagram showing the software functional configuration of the camera 101. The following describes the processing performed when capturing an image by the camera 101 having the functional configuration shown in Fig. 3. Note that the following description focuses on the processing performed by each functional unit shown in Fig. 3.

[0027] The following describes the functions of the camera 101 realized by the blocks shown in Fig. 3. The camera 101 has an image acquisition unit 301, an object detection unit 302, an object movement estimation unit 303, a template storage unit 304, a template search unit 305, a control instruction reception unit 306, a shooting mode switching unit 307, a template reproduction control unit 308, an automatic shooting control unit 309, and a field of view change unit 310.

[0028] The CPU 201 may execute a computer program to realize the above-described functional units of the camera 101. The CPU 201 may execute a computer program to realize some of the functional units of the camera 101. The camera 101 may realize some or all of the functional units using the CPU 201 and a processor other than the CPU 201. The camera 101 may realize some or all of the functional units using hardware such as one or more circuits, such as an ASIC and a PLD including an FPGA.

[0029] The image acquisition unit 301 converts the electrical signal acquired from the image sensor into image data and stores the converted image data in the RAM 202 of the camera 101. Note that the image acquisition unit 301 may also acquire the converted image data and store it in the RAM 202.

[0030] The object detection unit 302 detects an object (also referred to as a subject) in the image by analyzing the image acquired by the image acquisition unit 301. One example of a method for detecting an object is a method in which an object is detected and recognized in an image by image recognition using a neural network. The object detection unit 302 detects and outputs at least one of the type of the detected object, the direction of the object in the image, the position of the object, and the size of the object. The type of object, the direction of the object in the image, the position of the object, and the size of the object are examples of information about the object. The object detection unit 302 may detect an object in a zoomed-out state, i.e., in a state in which many objects are included in the image.

[0031] The object motion estimation unit 303 estimates and identifies the motion of moving objects, including humans, animals, drones, etc., among the objects detected by the object detection unit 302, through image analysis. The object motion estimation unit 303 may identify the motion through skeletal estimation and captioning using image recognition that utilizes a neural network, but the method for identifying the motion is not particularly limited. The object motion estimation unit 303 outputs information that identifies the object, and the presence and type of motion performed by the object.

[0032] The template storage unit 304 stores information on one or more preset templates (also referred to as template information). The template storage unit 304 may store a list of multiple pieces of template information in table format. The template information includes information defining the arrangement of objects as information for reproducing the angle of view (also referred to as composition). The template information includes, for example, at least one of information on the type (also referred to as variety) of an object, the orientation, position, and size of an object in an image, and information defining the range of each piece of information. A template may include information on multiple object types. Details of the information included in the template information and the composition information and the like stored in the template will be described later.

[0033] The template search unit 305 searches for a template applicable to controlling the angle of view of the camera 101 from multiple templates containing information about predetermined object locations. For example, the template search unit 305 searches the template storage unit 304 for a template applicable to controlling the angle of view of the currently captured image based on information output by the object detection unit 302. A template applicable to controlling the angle of view can also be said to be a template that can perform PTZ control on objects in the image. The template search unit 305 may search for templates stored in the template storage unit 304 that include objects whose object types are all detected from the image by the object detection unit 302. When multiple templates are suitable, the template search unit 305 may determine the template to be executed based on the template priority assigned to the templates.

[0034] The control instruction receiving unit 306 transmits PTZ parameters, which are signals for realizing PTZ control of the camera 101, to the angle of view changing unit 310 in accordance with the control instruction received from the operation input device 103. Furthermore, when a control instruction has been received, the control instruction receiving unit 306 notifies the shooting mode switching unit 307 that the control instruction has been received.

[0035] The shooting mode switching unit 307 switches the shooting mode depending on the search results of the template search unit 305 and the reception status of control instructions and the like notified from the control instruction receiving unit 306. In this embodiment, the shooting mode switching unit 307 determines which shooting mode to use for shooting: automatic shooting mode, template shooting mode, or manual shooting mode. When the control instruction receiving unit 306 receives a control instruction from the operation input device 103, the shooting mode switching unit 307 determines the shooting mode to be the manual control mode in which the PTZ is controlled in accordance with the control signal. When the control instruction receiving unit 306 does not receive a control instruction and the template search unit 305 has output a template applicable to the image, the shooting mode switching unit 307 determines the shooting mode to be the template shooting mode in which the angle of view is controlled by PTZ control based on the template. When the control instruction receiving unit 306 does not receive a control instruction and the template search unit 305 has not searched for an applicable template, the shooting mode switching unit 307 determines the shooting mode to be the automatic shooting mode in which the angle of view is automatically controlled to automatically track an object in the image.

[0036] When the shooting mode switching unit 307 determines the template shooting mode as the shooting mode, the template reproduction control unit 308 sets PTZ parameters for adjusting the angle of view to achieve the composition specified in the template searched for by the template search unit 305. The template reproduction control unit 308 outputs the PTZ parameters for adjusting the angle of view to the angle of view changing unit 310. The method for adjusting the angle of view will be described later. The PTZ parameters may also be referred to as PTZ coordinates.

[0037] The automatic shooting control unit 309 generates PTZ parameters for automatically controlling the angle of view based on the analysis results of the captured image and outputs them to the angle of view change unit 310. In other words, the automatic shooting control unit 309 generates PTZ parameters for realizing PTZ control that enables automatic tracking and shooting of an object in an image based on the analysis results and outputs them to the angle of view change unit 310. The analysis results include at least one of the type, direction, position, and size of the object detected by the object detection unit 302, and the movement speed and movement direction of the object estimated by the object movement estimation unit 303. Methods for automatically tracking an object may generally include a method using a neural network to detect movement or specific parts of an object, a method using template matching, or the like. The automatic tracking method is not particularly limited.

[0038] The angle-of-view changing unit 310 controls at least one of panning, tilting, and zooming to change and control the angle of view. Specifically, the angle-of-view changing unit 310 controls hardware such as the motor 208 in accordance with PTZ parameters received from any of the control instruction receiving unit 306, the template reproduction control unit 308, and the automatic shooting control unit 309, to perform PTZ control to change the angle of view of the camera 101. For example, the angle-of-view changing unit 310 may control at least one of panning, tilting, and zooming of the camera 101 to change and control the angle of view based on the arrangement of an object specified in the searched template. The angle-of-view changing unit 310 may also realize pseudo-PTZ control by using electronic zoom and cropping out a portion of an image.

[0039] The template information table will be described using Figures 4 and 5. The template information table is a list of one or more pieces of template information. Figure 4 is a diagram showing an example of a template information table for controlling the angle of view stored in template storage unit 304 of camera 101. Figure 5 is a diagram explaining the orientation range, position range, and size range included in template information. The template information table is stored in template storage unit 304 of camera 101.

[0040] 4, the template information table in this embodiment includes at least group, template ID, template priority, type, object priority, orientation range, position range, and size range. Each item in the template information table will be described. The orientation range, position range, and size range are examples of information related to the placement of an object.

[0041] A group may be information that defines the location where the image is taken, the shooting scene, etc. A group may include one or more templates. For example, if the operator specifies a group when taking an image, the template search unit 305 can limit the search to templates that belong to the specified group, allowing the operator to take a more suitable image.

[0042] The template ID is an identifier for identifying each template, and may be information unique to each template.

[0043] The template priority is priority information that indicates which template should be given priority for controlling the angle of view when the template search unit 305 determines, based on the type of object in the image, that multiple templates are applicable to controlling the angle of view using PTZ control.

[0044] The type represents the type of object included in the field of view. For example, the type may be the name of the object, such as the person and podium shown in FIG. 4. The type may also be a part of the person's body, such as the upper body or face, or a part of the object. The template search unit 305 searches for a template in which the type of the object included in the image matches the type included in the template information.

[0045] The object priority is information indicating the priority for determining which object, among the objects included in the angle of view, should be given priority in adjusting the angle of view in terms of its position, orientation, size, etc. Therefore, when there are multiple objects of different types in an image, the template search unit 305 searches for templates according to the object priority.

[0046] The orientation range represents the range of orientations of an object when reproducing the composition defined in the template. For example, in the case of a person, the orientation range is information that defines in what direction the front direction of the body, estimated from the orientation and posture, should face relative to the imaging surface of the camera. FIG. 5(a) is a diagram illustrating an example of the orientation range. Here, an example of the orientation range in the upper row for template ID "A" shown in FIG. 4 is described. As shown in FIG. 5(a), the orientation of the face of person 504 facing the imaging surface of camera 101 is set to 0 degrees. The angle-of-view changing unit 310 performs PTZ control to capture an image so that the orientation of the face falls within the angle line 509 to angle line 510, which indicate degS1 to degE1 degrees.

[0047] The position range is expressed in two-dimensional coordinates and is information indicating the range within which the position coordinates of an object are contained. For example, the position coordinates of an object indicate the vertices of a rectangle (bounding box) that represents the object region in an image. Generally, the coordinates of the upper left vertex of the rectangle are used as the position coordinates of the object, but any coordinates that can uniquely identify the position among all objects in each template are sufficient. FIG. 5(b) is a diagram illustrating an example of the position range. An example of the position range in the upper row of FIG. 4 for template ID "A" is described below. As shown in FIG. 5(b), a rectangle 508 indicates the face region of a person 504 present within a photographing angle of view 501. The position range here defines a vertex 506 present at coordinates (SX1, SY1) and a vertex 505 present at coordinates (EX1, EY1). The angle of view change unit 310 performs PTZ control to photograph an image so that the upper left coordinate 507 of the rectangle 508 of the face region is within an area 511 defined by the vertices 505 and 506.

[0048] The size range is information indicating the range of the size of the object region in the image. The angle of view changer 310 performs PTZ control so that the entire rectangle representing the object region fits within the range specified by the size range. FIG. 5(c) is a diagram illustrating an example of the size range. An example of the size range in the upper row of FIG. 4 for template ID "A" will be described. As shown in FIG. 5(c), area 502 is defined by the height SH1 and width SW1 of the size range. Area 503 is defined by the height EH1 and width EW1 of the size range. The angle of view changer 310 performs PTZ control to capture an image so that the size of a rectangle 508 representing the face region of person 504 is larger than area 502 and smaller than area 503.

[0049] The template search unit 305 searches the template information table shown in Fig. 4 for a template that can be applied to controlling the angle of view by PTZ control, based on the image currently being captured. An applicable template can also be said to be a template that can be implemented to control the angle of view of the image currently being captured. The template search unit 305 may search for an applicable template by comparing the type, orientation, size, and position of an object specified in the template with the type, orientation, size, and position of an object captured in the captured image.

[0050] For example, if one or all of the types specified in a template match the types of objects present in the image, the template search unit 305 may determine that the template is applicable to controlling the angle of view through PTZ control. Also, if different types of objects exist in the image, the template search unit 305 may search for a template based on an object with a high object priority.

[0051] The template search unit 305 may determine whether the orientation, position, and size of an object in an image being captured fall within the orientation range, position range, and size range specified in the template, and determine whether the template is applicable. Specifically, the template search unit 305 may determine that the template is applicable if the orientation, position, and size of an object in an image being captured fall within the orientation range, position range, and size range specified in the template. However, the template search unit 305 may also determine that the template is applicable if the angle of view change unit 310 controls the PTZ of the camera 101 so that the orientation, position, and size of an object in an image fall within the range specified in the template.

[0052] If there are multiple templates determined to be applicable based on a comparison of types, etc., the template search unit 305 may select the template with the highest template priority and output it as the search result. If multiple templates with the same template priority are searched for, the template search unit 305 may output the template that has the smallest difference from the composition of the currently captured image as the search result. The difference in composition may be the difference between the orientation, position, and size of an object in the currently captured image and the composition resulting from PTZ control based on the orientation range, position range, and size range specified in the template.

[0053] Fig. 6 is a flowchart of the angle-of-view control process executed by the camera 101. The angle-of-view control process will be described with reference to the flowchart in Fig. 6. The CPU 201 executes a computer program for the angle-of-view control process to realize the functional units shown in Fig. 3 and execute each step of the angle-of-view control process. The camera 101 may implement the functional units shown in Fig. 3 in hardware.

[0054] The CPU 201 may continue the angle-of-view control process from step S601 to step S608 shown in FIG. 6 while capturing an image. The CPU 201 may synchronize the start and end of the angle-of-view control process with the start and stop of the camera 101, or with an instruction to enable template capture and automatic capture received from the operation input device 103. As preprocessing before executing the angle-of-view control process, the CPU 201 may set the zoom value to the wide-angle end, or may control panning or tilting to detect in advance an object present within a captureable range. By detecting an object in advance, the CPU 201 can limit the templates to be searched in advance, allowing for efficient template search, and may store in advance candidate angles of view to which templates can be applied.

[0055] In step S601, the control instruction receiving unit 306 checks whether a manual PTZ control signal is received from the operation input device 103. If a PTZ control signal is received, the control instruction receiving unit 306 notifies the shooting mode switching unit 307.

[0056] In step S602, the shooting mode switching unit 307 determines whether or not a manual control instruction has been received. If the shooting mode switching unit 307 determines that a manual control instruction has been received from the control instruction receiving unit 306, the process proceeds to step S6090. If the shooting mode switching unit 307 determines that a manual control instruction has not been received from the control instruction receiving unit 306, the process proceeds to step S603.

[0057] In step S6090, the shooting mode switching unit 307 switches the shooting mode to the manual shooting mode, and the process proceeds to step S609.

[0058] In step S603, the image acquisition unit 301 acquires the image captured by the camera 101.

[0059] In step S604, the object detection unit 302 analyzes the image acquired by the image acquisition unit 301 in step S603 and detects an object in the image. The object detection unit 302 detects and outputs at least one of the type of the detected object, the orientation of the object in the image, the position of the object, and the size of the object as information about the object.

[0060] In step S605, the template search unit 305 searches for templates that can be applied to the angle of view of the image currently being captured. Specifically, the template search unit 305 searches for one or more applicable templates from the template information table held by the template holding unit 304, based on information about the object detected by the object detection unit 302 in step S604. If multiple templates are found by the search, the template search unit 305 selects and outputs the template with the highest template priority as the template to be used for controlling the angle of view.

[0061] In step S606, if the template search unit 305 is able to search for a template that can be applied to the current angle of view of shooting, it outputs a message to that effect to the shooting mode switching unit 307 and proceeds to step S6070. If the template search unit 305 is unable to search for an applicable template, it proceeds to step S6110.

[0062] In step S6070, the photographing mode switching unit 307 switches the photographing mode to the template photographing mode, and the process proceeds to step S607.

[0063] In step S607, the object motion estimation unit 303 estimates whether any of the objects detected in step S604 is moving, and the type of moving object specified in the template is moving. If the object motion estimation unit 303 estimates that the moving object specified in the template is moving, it notifies the shooting mode switching unit 307 that the object is moving. However, if the object motion estimation unit 303 determines that the object does not deviate from the orientation range, position range, and size range specified in the template based on the object's movement speed and movement direction, etc., the object motion estimation unit 303 does not need to notify that the object is moving. Furthermore, if an object not specified in the template is moving, the object motion estimation unit 303 does not need to notify that the object is moving. This allows the object motion estimation unit 303 to prevent unintended switching of the shooting mode when an object unrelated to the reproduction of the composition is moving.

[0064] In step S608, if the shooting mode switching unit 307 has received a notification from the object motion estimation unit 303 that the object is moving, the process proceeds to step S6110. If the shooting mode switching unit 307 has not received a notification from the object motion estimation unit 303, the process proceeds to step S610.

[0065] In step S609 , the control instruction receiving unit 306 calculates and generates PTZ parameters that define the speed, distance, and direction based on the control signal received from the operation input device 103 , and outputs the PTZ parameters to the angle of view changing unit 310 .

[0066] In step S610, the template reproduction control unit 308 calculates and generates PTZ parameters so that the composition specified in the template output by the template search unit 305 is achieved, and outputs the PTZ parameters to the angle of view change unit 310.

[0067] In step S6110, the shooting mode switching unit 307 switches the shooting mode to the automatic shooting mode.

[0068] In step S611, the automatic shooting control unit 309 calculates and generates PTZ parameters that define the direction and speed for automatically tracking a moving object based on the analysis results, and outputs the PTZ parameters to the angle of view changing unit 310. For example, the automatic shooting control unit 309 may generate the PTZ parameters based on at least one of the type, direction, position, and size of the object detected by the object detection unit 302 and the moving speed and moving direction of the object estimated by the object motion estimation unit 303. The automatic shooting control unit 309 may set the moving object to be tracked based on information specified by the operator via the operation input device 103.

[0069] In step S612, the angle of view changing unit 310 controls the angle of view in accordance with the shooting mode determined by the shooting mode switching unit 307 in step S6090, step S6070, or step S6110. At that time, the angle of view changing unit 310 performs PTZ control of the camera 101 in accordance with PTZ parameters generated by any one of the control instruction receiving unit 306, template reproduction control unit 308, or automatic shooting control unit 309.

[0070] [Effects of the embodiment] As described above, the camera 101 of the embodiment executes PTZ control by searching for a template applicable to controlling the angle of view through PTZ control. This reduces the burden on the operator required for searching for a template, and enables automatic control of the angle of view based on the searched template.

[0071] In this embodiment, the composition is determined according to a template and the angle of view is automatically controlled, and even if the template is not established, the system continues shooting automatically, thereby making it possible to take images with reduced burden on the operator.

[0072] This embodiment can be applied to applications such as live streaming and video production of online events, which have seen increasing demand in recent years, for example, entertainment video shooting of music events, plays, sporting events, etc. Even when applied to such applications, this embodiment can achieve the above-mentioned effects.

[0073] This embodiment detects information about an object included in an image, including any of the type, orientation, position, and size of the object, and searches for an applicable template based on the information about the object. This allows this embodiment to accurately search for an applicable template.

[0074] This embodiment is configured to be switchable among template shooting mode, automatic shooting mode, and manual shooting mode, so that even if an applicable template cannot be found, the embodiment can track and shoot an object in another shooting mode.

[0075] In this embodiment, if a template is found, the template photographing mode is automatically set, so that the burden on the operator in setting the photographing mode can be further reduced.

[0076] In this embodiment, even if a template can be searched for, if a manual control instruction is received from the operator, the mode is switched to manual shooting mode, so that shooting according to the operator's wishes can be realized.

[0077] In this embodiment, if the template shooting mode or manual shooting mode cannot be set, the automatic shooting mode is automatically set, thereby further reducing the burden on the operator in setting the shooting mode. Also, in this embodiment, if there is no applicable template, the automatic shooting mode is set, so even if there is no template applicable to control of the angle of view by PTZ control, shooting can be automatically continued and the object can be automatically tracked.

[0078] In this embodiment, a template is searched for so that the type of object in the template matches the type of object contained in the image, and therefore the angle of view can be controlled using a template appropriate for the object in the image.

[0079] This embodiment searches for a template in which the orientation, position, and size of an object in an image are within at least one of the orientation range, position range, and size range of the object in the template, thereby enabling the angle of view to be controlled using a template that can more reliably track the object in the image.

[0080] In this embodiment, since the template includes a template priority, even if there are multiple templates that can be applied to control the angle of view, it is possible to search for an appropriate template.

[0081] In this embodiment, since the template includes an object priority, even if a plurality of objects exist in an image, it is possible to search for a template suitable for an object that meets the operator's desire.

[0082] In this embodiment, if it is determined that multiple templates are applicable as a result of the template search, the template that minimizes the difference between the image composition and the composition changed by the template is applied to control the angle of view, thereby reducing changes in the angle of view due to template control.

[0083] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0084] The disclosure of this specification includes the following view angle control device, view angle control method, and program. (Item 1) A field of view control device for controlling a field of view of an imaging device, a template search means for searching for a template applicable to the control of the angle of view from a plurality of templates each including information on the arrangement of a predetermined object; a view angle changing means for changing and controlling the view angle by controlling at least one of panning, tilting, and zooming of the image capturing device based on the arrangement of the object specified in the searched template; A field angle control device comprising: (Item 2) The template includes information on at least one of the type of the object, the orientation of the object, the position of the object, and the size of the object. 2. The field angle control device according to item 1, (Item 3) image acquisition means for acquiring a photographed image captured by the photographing device; an object detection means for detecting at least one of the type, direction, position, and size of the object included in the captured image as information about the object; The template search means searches for a template that can be applied to control the angle of view based on information about the object. 3. The field angle control device according to item 1 or 2, characterized in that (Item 4) a photographing mode switching means for setting the photographing mode to one of a first photographing mode in which the angle of view is controlled based on the template, a second photographing mode in which the angle of view is manually controlled by an operator, and a third photographing mode in which the angle of view is automatically controlled based on an analysis result of an image photographed by the photographing device, The angle of view changing means controls the angle of view based on the shooting mode. 4. The field angle control device according to any one of items 1 to 3, characterized in that: (Item 5) The photographing mode switching means sets the photographing mode to the first photographing mode when the template searching means has searched for the template. 5. The field angle control device according to item 4. (Item 6) When the photographing mode switching means receives a control signal for controlling the angle of view, the means sets the second photographing mode regardless of the search result of the template. 6. The field angle control device according to item 4 or 5, (Item 7) The photographing mode switching means sets the third photographing mode when neither the first photographing mode nor the second photographing mode can be set. 7. The field angle control device according to claim 4, wherein the field angle control device is a field angle control device. (Item 8) the template includes, as template information, at least any one of information on a type of the object, a range of orientations of the object, a range of positions of the object, and a range of sizes of the object; The template search means searches for the template including the object type included in the template information that matches the object type included in the information about the object. 4. The field angle control device according to item 3. (Item 9) The template search means searches for a template in which at least one of the direction, position, and size of the object included in the information about the object is within at least one of the range of the orientation of the object, the range of the position of the object, and the range of the size of the object included in the template information. 9. The field angle control device according to item 8. (Item 10) the template includes a template priority indicating a priority of the template; When the template search means determines that a plurality of templates are applicable, the template search means selects one template from the plurality of applicable templates based on the template priority. 10. The field angle control device according to item 8 or 9, characterized in that (Item 11) the template includes an object priority indicating a priority of the object; When the information about the object includes a plurality of object types, the template search means searches for the template based on the object having a high object priority. 11. The field angle control device according to any one of items 8 to 10, characterized in that: (Item 12) When it is determined that a plurality of templates are applicable, the template search means selects the template that has the smallest difference between the composition of the photographed image and the composition of the angle of view changed by the template as a search result. 12. The field angle control device according to any one of items 8 to 11, wherein: (Item 13) The photographing mode switching means sets the photographing mode to the first photographing mode when the template search means has searched for the template and when the object is not moving. 5. The field angle control device according to item 4. (Item 14) The photographing mode switching means sets the photographing mode to the first photographing mode when the template search means has searched for the template and when the object is moving within a range of the object arrangement indicated by the template. 5. The field angle control device according to item 4. (Item 15) A method for controlling an angle of view of an imaging device, comprising: a template search step of searching for a template applicable to the control of the angle of view from a plurality of templates each including information on the arrangement of a predetermined object; a view angle changing step of changing the view angle by controlling at least one of panning, tilting, and zooming of the image capturing device based on the arrangement of the object specified in the searched template; A method for controlling an angle of view. (Item 16) A program for causing a computer to function as each means of the field of view control device according to any one of items 1 to 14.

[0085] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0086] 101: Camera; 103: Operation input device; 201: CPU; 301: Image acquisition unit; 302: Object detection unit; 305: Template search unit; 307: Shooting mode switching unit; 310: View angle changing unit.

Claims

1. A field of view control device for controlling a field of view of an imaging device, a template search means for searching for a template applicable to the control of the angle of view from a plurality of templates each including information on the arrangement of a predetermined object; a view angle changing means for changing and controlling the view angle by controlling at least one of panning, tilting, and zooming of the image capturing device based on the arrangement of the object specified in the searched template; A field angle control device comprising:

2. The template includes information on at least one of the type of the object, the orientation of the object, the position of the object, and the size of the object.

2. The field angle control device according to claim 1,

3. image acquisition means for acquiring a photographed image captured by the photographing device; an object detection means for detecting at least one of the type, direction, position, and size of the object included in the captured image as information about the object; The template search means searches for a template that can be applied to control the angle of view based on information about the object.

2. The field angle control device according to claim 1,

4. an imaging mode switching means for setting the imaging mode to one of a first imaging mode in which the angle of view is controlled based on the template, a second imaging mode in which the angle of view is controlled manually by an operator, and a third imaging mode in which the angle of view is automatically controlled based on an analysis result of an image captured by the imaging device; The angle of view changing means controls the angle of view based on the shooting mode.

2. The field angle control device according to claim 1,

5. The photographing mode switching means sets the photographing mode to the first photographing mode when the template searching means has searched for the template.

5. The field angle control device according to claim 4,

6. When the photographing mode switching means receives a control signal for controlling the angle of view, the means sets the second photographing mode regardless of the search result of the template.

5. The field angle control device according to claim 4,

7. The photographing mode switching means sets the third photographing mode when neither the first photographing mode nor the second photographing mode can be set.

5. The field angle control device according to claim 4,

8. the template includes, as template information, at least any one of information on a type of the object, a range of orientations of the object, a range of positions of the object, and a range of sizes of the object; The template search means searches for the template including the object type included in the template information that matches the object type included in the information about the object.

4. The field angle control device according to claim 3,

9. The template search means searches for a template in which at least one of the direction, position, and size of the object included in the information about the object is within at least one of the range of the orientation of the object, the range of the position of the object, and the range of the size of the object included in the template information.

9. The field angle control device according to claim 8,

10. the template includes a template priority indicating a priority of the template; When the template search means determines that a plurality of templates are applicable, the template search means selects one template from the plurality of applicable templates based on the template priority.

9. The field angle control device according to claim 8,

11. the template includes an object priority indicating a priority of the object; When the information about the object includes a plurality of object types, the template search means searches for the template based on the object having a high object priority.

9. The field angle control device according to claim 8,

12. When it is determined that a plurality of templates are applicable, the template search means selects the template that has the smallest difference between the composition of the photographed image and the composition of the angle of view changed by the template as a search result.

9. The field angle control device according to claim 8,

13. The photographing mode switching means sets the photographing mode to the first photographing mode when the template searching means has searched for the template and when the object is not moving.

5. The field angle control device according to claim 4,

14. The photographing mode switching means sets the photographing mode to the first photographing mode when the template search means has searched for the template and when the object is moving within a range of the object arrangement indicated by the template.

5. The field angle control device according to claim 4,

15. A method for controlling an angle of view of an imaging device, comprising: a template search step of searching for a template applicable to the control of the angle of view from a plurality of templates each including information on the arrangement of a predetermined object; a view angle changing step of changing the view angle by controlling at least one of panning, tilting, and zooming of the image capturing device based on the arrangement of the object specified in the searched template; A method for controlling an angle of view.

16. A program for causing a computer to function as each means of the field angle control device according to any one of claims 1 to 14.

Images