Imaging control device, control method, and program

Abstract

To control the angle of view of a camera to allow distribution of images suitable for a subject.SOLUTION: An imaging control device is associated with an imaging apparatus that can control the angle of view to track a subject according to a parameter indicating the sensitivity of control of the angle of view relative to the position of the subject. The imaging control device acquires distribution information indicating a distribution state of images picked up by the imaging apparatus. The imaging control device sets the parameter for the imaging apparatus on the basis of the distribution information and positional information indicating the position of the subject in the images.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging control device, a control method, and a program, and more particularly to distribution of video captured while tracking a subject. [Background technology]

[0002] In a video distribution system, a technique is known in which a switcher is used to switch the video to be distributed among the videos from a plurality of cameras. Patent Document 1 discloses a technique in which, when a video obtained by a camera is being distributed, the camera does not operate according to an external control command. The technique of Patent Document 1 makes it possible to prevent the imaging direction of the video being distributed from being changed by the controller contrary to the intention of the switcher. Patent Document 2 discloses a technique in which, when a video obtained by a camera is being distributed, the driving speed of the pan or tilt operation of the camera is limited. The technique of Patent Document 2 makes it possible to prevent image stuttering in the video being distributed, even when a pan or tilt operation is performed according to an external control command.

[0003] In recent years, automatic tracking shooting has been introduced, where a desired subject is detected using artificial intelligence (AI) technology that enables real-time object detection, and the subject is automatically tracked. Users can broadcast by switching between images from cameras that are automatically tracking using a switcher. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-45529 [Patent Document 2] Patent Publication No. 2021-34824 Summary of the Invention [Problem to be solved by the invention]

[0005] In automatic tracking, there is a possibility that the subject may move out of the angle of view due to a sudden movement of the subject. If the driving speed of the pan-tilt-zoom (PTZ) operation of the camera is limited while the image captured by the camera is being distributed using the technology of Patent Document 2, the possibility of the subject moving out of the angle of view increases. On the other hand, if the driving speed is not limited, the stuttering of the distributed image becomes large.

[0006] An object of the present invention is to control the angle of view of a camera so as to deliver an appropriate image of a subject. [Means for solving the problem]

[0007] An imaging control device according to an embodiment of the present invention has the following configuration. An imaging control device associated with an imaging device capable of controlling an angle of view so as to track an object according to a parameter indicating sensitivity of angle of view control to a position of the object, An acquisition means for acquiring distribution information indicating a distribution status of the video captured by the imaging device; a setting means for setting the parameters for the imaging device based on the distribution information and position information indicating a position of the subject in the video; Equipped with. Effect of the Invention

[0008] An object of the present invention is to control the angle of view of a camera so as to deliver an appropriate image of a subject. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing an example of the overall configuration of an automatic tracking imaging system according to an embodiment; [Diagram 2] FIG. 1 is a diagram showing an example of the hardware configuration of an automatic tracking imaging system according to an embodiment. [Diagram 3] FIG. 1 is a diagram showing an example of the functional configuration of an automatic tracking imaging system according to an embodiment. [Figure 4]FIG. 13 is a diagram showing an example of the relationship between a difference amount and a control amount. [Diagram 5] 1A and 1B are diagrams illustrating subject detection using a learning model. [Figure 6] 4 is a flowchart of an imaging control method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] 1 shows an example of the configuration of an automatic tracking photography system according to an embodiment. This system includes an information processing device 100, an imaging device 200, an operation device 300, a switching device 400, and a network 500. A user operates the operation device 300. Then, a command corresponding to the operation is transmitted to the imaging device 200 via the information processing device 100 and the network 500. The imaging device 200 performs an operation corresponding to the received command. In this manner, the user operates the imaging device 200. This system may include multiple imaging devices 200 for photography.

[0012] An image captured by the imaging device 200 is input to the information processing device 100. The information processing device 100 performs various calculations and recordings required for automatic tracking shooting. The operation device 300 transmits setting commands for setting various parameters to the information processing device 100. In this manner, imaging settings for automatic tracking shooting are controlled.

[0013] A plurality of video signals are input to the switching device 400. Each of the plurality of video signals may be input by each of the plurality of imaging devices 200. The switching device 400 can select a video to be output for distribution (hereinafter referred to as a distribution video) from the plurality of video signals. The switching device 400 can distribute the distribution video to any device via the network 500. The switching device 400, the imaging device 200, and the information processing device 100 are connected to each other so as to be able to communicate with each other via the network 500. A user can select a distribution video using the switching device 400. The switching device 400 can transmit distribution information indicating a distribution state of a video captured by the imaging device 200 to the information processing device 100. This distribution information may indicate a video selected as a distribution video among the videos captured by the plurality of imaging devices 200. The distribution information may indicate, for each of the plurality of imaging devices 200, whether or not a video captured by each imaging device 200 is selected as a distribution video. The network 500 is a communication line such as a public telephone line or the Internet.

[0014] The configuration of the distribution system is not limited to that shown in Fig. 1. For example, the operation device 300 may be directly connected to the network 500. A user may control the switching device 400 to select a distribution video designated by the user via a device connected to the network 500. Two or more of the information processing device 100, the imaging device 200, the operation device 300, and the switching device 400 may be connected via a communication path such as a cable without the network 500.

[0015] Fig. 2 shows an example of the hardware configuration of the automatic tracking photography system shown in Fig. 1. The information processing device 100 has a RAM 101, a GPU 102, a CPU 103, an input unit 104, a storage unit 105, a communication unit 106, and a UI unit 107. The imaging device 200 has a camera 201, a drive unit 202, a communication unit 203, a CPU 204, and a storage unit 205. The operation device 300 has a communication unit 303, an operation unit 302, a storage unit 301, a CPU 304, and a display unit 305. The switching device 400 has a RAM 405, a CPU 406, a video input unit 401, a video output unit 402, an operation unit 404, and a communication unit 403.

[0016] First, a hardware configuration example of the information processing device 100 will be described. The RAM 101 is a volatile memory. The CPU 103 is a processor that controls each unit of the information processing device 100. The CPU 103 performs processing according to a program stored in, for example, the storage unit 105. The CPU 103 can use the RAM 101 as a work memory. The GPU 102 is a processor that performs arithmetic processing. The GPU 102 can perform efficient calculations by parallel processing. For this reason, by using the GPU 102, it is possible to speed up learning using a learning model with many parameters such as a deep neural network.

[0017] The input unit 104 is an interface for inputting a video signal to the information processing device 100, and is capable of wirelessly inputting a video signal via, for example, a network 500. The input unit 104 is also a variety of communication interfaces such as a wired network. The input unit 104 can be, for example, a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI (registered trademark)), or a Serial Digital Interface (SDI).

[0018] The storage unit 105 is a non-volatile memory. The storage unit 105 can store data such as image data and various programs used by the CPU 103 in respective predetermined areas. The storage unit 105 can be a storage medium such as a Hard Disk Drive (HDD) or a flash memory.

[0019] The communication unit 106 is a communication interface connected to the network 500. The communication unit 106 can communicate with the imaging device 200 or the operation device 300 via the network 500 under the control of the CPU 103. The UI unit 107 is a user interface. The UI unit 107 can accept operation input from a user for operating the information processing device 100. The UI unit 107 can also present information to the user. The UI unit 107 can be a keyboard, a mouse, a display, or a touch panel.

[0020] In this way, the functions of the information processing device 100, which is an imaging control device according to an embodiment, can be realized by a computer having a processor and a memory. For example, a processor such as a CPU 103 executes a program stored in a memory such as a RAM 101 or a storage unit 105, thereby realizing the functions of each unit shown in FIG. 3, which will be described later. Note that the functions of the information processing device 100 shown in FIG. 3 can be realized by a computer, but some or all of the functions of the information processing device 100 may be realized by dedicated hardware. In addition, the information processing device 100 may be composed of a plurality of information processing devices connected via a network, for example. The same applies to the imaging device 200, the operation device 300, and the switching device 400.

[0021] Next, an example of the hardware configuration of the imaging device 200 will be described. The camera 201 captures an image. The camera 201 can capture an image of the surroundings of the imaging device 200. In this way, the camera 201 can capture an image of a target subject. Here, the angle of view of the imaging device 200 is controllable. For example, the imaging device 200 can perform at least one of a panning operation, a tilting operation, and a zooming operation. The camera 201 may have an optical zoom lens capable of changing the imaging magnification. In this case, the CPU 204 described later transmits a zoom control command value to the camera 201, thereby changing the magnification of the captured image. The camera 201 may also have a digital zoom function that locally enlarges a part of the captured image. The digital zoom can be used when the magnification is insufficient with the optical zoom, for example, when it is desired to further enlarge the captured image.

[0022] The driving unit 202 performs a panning operation or a tilting operation of the imaging device 200. The driving unit 202 may include, for example, an actuator for rotating in a panning direction, a tilting direction, or both directions, as well as a driving circuit and a peripheral circuit for the actuator. In this manner, the driving unit 202 can rotate the imaging device 200 in the panning direction and the tilting direction.

[0023] The communication unit 203 is an interface for connecting to the network 500. The communication unit 203 is connected to the communication unit 106 of the information processing device 100 via the network 500. The communication unit 203 can communicate with the information processing device 100 under the control of the CPU 204. The communication unit 203 is also connected to the input unit 104 of the information processing device 100 and the video input unit 401 of the switching device 400 via the network 500. The communication unit 203 can output a video signal obtained by imaging to the information processing device 100 and the switching device 400.

[0024] The CPU 204 controls each unit of the imaging device 200. For example, the CPU 204 can perform processing according to a program stored in the storage unit 205. The storage unit 205 is a non-volatile memory. The storage unit 205 can store data such as setting data of the imaging device 200 and various programs used by the CPU 204 in predetermined areas.

[0025] In this embodiment, the imaging device 200 can control the angle of view so as to track the subject. For example, by the operation of the driving unit 202, the imaging device 200 can change the shooting direction so as to track the subject. Also, the camera 201 can change the shooting magnification so as to track the subject. In this way, the imaging device 200 can capture a tracking video of the target subject. The information processing device 100 can control the angle of view of the imaging device 200 so as to automatically track the subject in this way. On the other hand, the operation of the imaging device 200 may be manually controlled. In this case, the imaging device 200 can control the angle of view according to a user instruction input using the operation device 300.

[0026] Next, a hardware configuration example of the operation device 300 will be described. The storage unit 301 is a non-volatile memory. The storage unit 301 can store data such as setting data of the operation device 300 and various programs used by the CPU 304 in a predetermined area. The operation unit 302 is an interface used by the operator to input instructions. The operation unit 302 can be, for example, a joystick, an operation lever, or various switches. The operator can perform rotation control (pan control and tilt control) and zoom control of the imaging device 200 by operating the operation unit 302. In addition, the operator can perform other controls of the imaging device 200, such as focus control, gain adjustment, and automatic tracking setting, by operating the operation unit 302. Furthermore, the operator can start or end automatic tracking shooting using the imaging device 200 by operating the operation unit 302.

[0027] The communication unit 303 is an interface for connecting to the network 500. The communication unit 303 can communicate with the information processing device 100 via the network 500 under the control of the CPU 304.

[0028] The CPU 304 controls each unit of the operation device 300. The CPU 304 can perform processing according to a program stored in the storage unit 301, for example. The display unit 305 is an interface that notifies the user of information. The display unit 305 can be, for example, an LED or a display. The display unit 305 can notify the user of the status of the imaging device 200, warnings, etc.

[0029] Next, a hardware configuration example of the switching device 400 will be described. The video input unit 401 is an interface for receiving video. The video input unit 401 can receive video from the imaging device 200 via the network 500. The video input unit 401 can be a wireless interface or a wired interface. The video input unit 401 may also be an interface compatible with USB, HDMI (registered trademark), SDI, or the like. The video input unit 401 can receive a plurality of video signals. Each of the plurality of video signals may be transmitted from a different imaging device 200.

[0030] The video output unit 402 is an interface for outputting a distribution video. The video output unit 402 can output a video input to the video input unit 401 via the network 500. The video output unit 402 can be a wireless interface or a wired interface. The video output unit 402 can also be an interface compatible with USB, HDMI (registered trademark), SDI, or the like. The communication unit 403 is an interface for communicating with an external device. The communication unit 403 can communicate with the information processing device 100 and the imaging device 200 via the network 500.

[0031] The operation unit 404 is used to perform an operation for selecting a video to be distributed from a plurality of videos input to the video input unit 401. For example, a user may select a video to be distributed by operating the operation unit 404. The RAM 405 is a volatile memory. The CPU 406 is a processor that controls each unit of the switching device 400. The CPU 406 can perform processing according to a program stored in the switching device 400 while using the RAM 405 as a working memory.

[0032] Each of the CPU 103, the CPU 204, the CPU 304, and the CPU 406 may be configured with one or more processors.

[0033] FIG. 3 shows a functional configuration example of an automatic tracking photography system according to an embodiment. First, a functional configuration example of an information processing device 100 that is an imaging control device according to an embodiment will be described. The information processing device 100 has an image processing unit 130, an estimation unit 131, a result processing unit 132, a movement prediction unit 133, an imaging control unit 134, a learning unit 135, a recording unit 136, and a distribution information acquisition unit 137. In this embodiment, the imaging device 200 can control the angle of view so as to track the subject according to a parameter indicating the sensitivity of the angle of view control to the position of the subject. The information processing device 100 is associated with such an imaging device 200. In the following embodiment, the information processing device 100 controls the angle of view of the imaging device 200 so as to track the subject. The information processing device 100 can perform this control according to a parameter indicating the sensitivity of the angle of view control to the position of the subject.

[0034] The image processing unit 130 acquires an image from the imaging device 200. The image processing unit 130 can perform signal processing on the acquired image. For example, the image processing unit 130 can resize or adjust the brightness of the image.

[0035] The estimation unit 131 and the result processing unit 132 acquire position information indicating the position of the subject in the video. First, the estimation unit 131 estimates the position of the subject in the video acquired from the imaging device 200. The estimation unit 131 can perform this estimation by inputting the video output by the image processing unit 130 to a learning model. In this example, the learning model generated by the learning unit 135 described later is used.

[0036] The result processing unit 132 performs various processes on the estimation result output by the estimation unit 131. For example, the result processing unit 132 can perform various noise processing or averaging processing. The image obtained by the imaging device 200 may contain noise other than the subject. The noise may be, for example, an object other than the tracking target or a part of the background. These may be mistaken for the subject. The result processing unit 132 can obtain a highly reliable subject position by removing the influence of these noises. In this way, the result processing unit 132 can output the position of the subject in the image (the subject's current position).

[0037] The estimation unit 131 and the result processing unit 132 may detect a specific type of subject. The estimation unit 131 and the result processing unit 132 may also detect a specific subject (e.g., a specific person). In this case, the imaging control unit 134 described later controls the imaging device 200 to track the detected subject according to the detection result.

[0038] The movement prediction unit 133 predicts the movement of the subject in the video. The movement prediction unit 133 can calculate the speed and traveling direction of the subject using the output result from the result processing unit 132. The movement prediction unit 133 can obtain the speed and traveling direction of the subject based on not only the output of the result processing unit 132 but also the angle of view (e.g., pan and tilt angles) of the imaging device 200. The movement prediction unit 133 can also perform statistical processing on the movement of the subject. In this way, the movement prediction unit 133 can predict the movement of the subject. The imaging control unit 134, which will be described later, may control the imaging device 200 to track the detected subject according to the movement prediction result.

[0039] The imaging control unit 134 sets a parameter indicating the sensitivity of the angle of view control to the position of the subject for the imaging device 200 based on the distribution information and the position information indicating the position of the subject in the video. In this specification, this parameter is called a tracking control parameter. The imaging control unit 134 can set the tracking control parameter for each of the multiple imaging devices 200.

[0040] Specifically, the imaging control unit 134 can set tracking control parameters for the imaging device 200 depending on whether or not the video captured by the imaging device 200 is being distributed. For example, when the distribution information indicates that the video captured by the imaging device 200 is being distributed, the imaging control unit 134 can set the tracking control parameters by a first method. Also, when the distribution information indicates that the video captured by the imaging device 200 is not being distributed, the imaging control unit 134 can set the tracking control parameters by a second method.

[0041] In addition, in this embodiment, the angle of view of the imaging device 200 can be controlled so that an image according to a predetermined composition is captured. This composition may represent a target position of a subject in the image. In this case, the angle of view of the imaging device 200 is controlled according to the composition. That is, the angle of view of the imaging device 200 can be controlled so that the subject approaches the target position in the image captured by the imaging device 200. Here, the imaging control unit 134 can further set the composition of the image based on the distribution information. The imaging control unit 134 may set the composition for the imaging device 200 depending on whether the image captured by the imaging device 200 is being distributed. For example, when the distribution information indicates that the image captured by the imaging device 200 is being distributed, the imaging control unit 134 can set the composition by the first method. Also, when the distribution information indicates that the image captured by the imaging device 200 is not being distributed, the imaging control unit 134 can set the composition by the second method.

[0042] Furthermore, the imaging control unit 134 may set the composition of the video based on the speed of the subject. The imaging control unit 134 may set at least one of the tracking control parameters and the composition of the video according to the type of the subject detected from the video. In this case, the estimation unit 131 can estimate the type of the subject in the video.

[0043] In this way, the tracking operation by the imaging device 200 can be changed depending on whether the video captured by the imaging device 200 is being distributed. For example, the angle of view of the imaging device 200 capturing the video being distributed can be controlled so that the composition of the video being distributed becomes closer to a desired composition and stuttering is reduced. On the other hand, the angle of view of the imaging device 200 capturing the video being distributed can be controlled so that the possibility of the subject falling out of the angle of view is reduced in preparation for the captured video being selected as the video being distributed. In this way, by changing the tracking operation depending on whether the video captured by the imaging device 200 is being distributed, the quality of the video distributed by the automatic tracking and photography system can be improved.

[0044] In addition, in this embodiment, the imaging control unit 134 controls the imaging device 200 to track the subject. The imaging control unit 134 can cause the imaging device 200 to change the angle of view at a speed according to the tracking control parameters so that the position of the subject in the video approaches the target position. In this embodiment, the imaging control unit 134 calculates the control amounts of pan and tilt based on the position of the subject in the video. A method of calculating the control amounts will be described later with reference to FIG. 4.

[0045] The learning unit 135 performs a learning process for the learning model used by the estimation unit 131. Details of the learning process will be described later with reference to FIG.

[0046] The recording unit 136 can record the video acquired from the imaging device 200. Furthermore, the recording unit 136 can record the learning data used by the learning unit 135 and the parameters of the learning model.

[0047] The distribution information acquisition unit 137 acquires distribution information indicating a distribution state of a video captured by an imaging device. The distribution information acquisition unit 137 can acquire the distribution information from the switching device 400. The distribution information acquisition unit 137 can acquire distribution information for each of the multiple imaging devices.

[0048] Next, a description will be given of an example of the functional configuration of the imaging device 200. The imaging device 200 includes a view angle control unit 230, a camera control unit 231, a setting management unit 232, and a communication unit 233.

[0049] The angle-of-view control unit 230 controls the angle of view of the camera 201. For example, the angle-of-view control unit 230 can output control signals for panning and tilting to the driving unit 202 based on a driving command received by the communication unit 233. Similarly, the angle-of-view control unit 230 can control the camera 201 to perform zooming. The camera control unit 231 outputs a signal for controlling the operation of the camera to the camera 201 based on a command received by the communication unit 233. The camera control unit 231 can control the camera 201 to start or end imaging, for example. The setting management unit 232 manages settings input in the operating device 300. Examples of settings include the maximum speed and drivable range of panning and tilting. The communication unit 233 exchanges commands and status information with the imaging control unit 134 according to a predetermined communication rule (protocol).

[0050] Next, a functional configuration example of the operation device 300 will be described. The operation device 300 has a communication unit 330 and a display unit 331. The communication unit 330 can exchange commands and status information with the imaging control unit 134 in accordance with a predetermined communication rule (protocol). The display unit 331 can display various information such as settings of the imaging device 200.

[0051] Next, a functional configuration example of the switching device 400 will be described. The switching device 400 has a communication unit 430 and a video switching unit 431. The communication unit 430 transmits distribution information to the distribution information acquisition unit 137. The video switching unit 431 switches the distribution video. As described above, the video switching unit 431 can select a distribution video from a plurality of videos in accordance with a user operation.

[0052] A method for controlling the angle of view for tracking a subject will be described with reference to FIG. 4. In the following example, the driving unit 202 performs panning and tilting in accordance with a control amount. This control amount can indicate the amount of movement of the panning and the amount of movement of the tilting. The driving unit 202 can rotate the imaging device 200 in the panning and tilting directions by an amount corresponding to such an amount of movement within a predetermined time. In this case, the control amount indicates the driving speed of the panning and the driving speed of the tilting.

[0053] In addition, the control amount of the pan operation and the tilt operation is determined according to the set tracking control parameter. In the following example, the tracking control parameter represents the tracking sensitivity (hereinafter referred to as tracking sensitivity). The tracking sensitivity is a parameter that controls the speed or acceleration of the angle of view control as follows.

[0054] In the automatic tracking, the angle-of-view control unit 230 of the imaging device 200 controls the driving unit 202 so that the target position in the image approaches the subject position. In the following example, the subject position is the center of gravity of the subject in the angle of view in the image. In order to perform such control, the imaging control unit 134 of the information processing device 100 calculates a control amount required to bring the target position closer to the subject position. For example, the imaging control unit 134 can calculate the difference from the target position to the subject position. Then, the imaging control unit 134 can calculate a control amount to eliminate this difference. The information processing device 100 transmits the control amount thus calculated to the imaging device 200. Then, the angle-of-view control unit 230 controls the driving unit 202 according to the transmitted control amount.

[0055] As an example, a case will be described in which the angle of view is controlled so as to maintain a composition in which the subject is located at the center of the angle of view of the video. In this case, the target position is the center of the angle of view. Here, the imaging control unit 134 detects the position of the center of the angle of view and the position of the subject, and calculates the difference. Then, according to the obtained difference, the imaging control unit 134 can calculate a control amount so as to perform a pan operation or tilt operation in a direction that eliminates this difference.

[0056] There are various methods for determining the magnitude of the control amount. In one embodiment, the tracking sensitivity may represent the magnitude of the control amount calculated by the angle of view control unit 230. In this case, the tracking sensitivity represents the drive speed of the pan operation and the tilt operation.

[0057] On the other hand, the magnitude of the control amount calculated by the imaging control unit 134 may be determined according to the tracking sensitivity and the magnitude of the difference from the target position to the subject position. For example, the control amount Y of the pan operation may be the product of the tracking sensitivity K and the difference amount X in the pan direction from the target position to the subject position, as shown in formula (1). In other words, when a predetermined tracking sensitivity K is set, the control amount Y is proportional to the difference amount X. Here, the tracking sensitivity K is variable. In this case, even if the subject position is constant, the control amount becomes larger as the tracking sensitivity becomes larger. Y = X × K …… Equation (1)

[0058] Also, the angle of view of the imaging device 200 may be controlled so that the angle of view of the imaging device 200 is not changed when the subject is included in a dead zone in the image. For example, when the difference between the target position and the subject position is within a specific range (hereinafter referred to as the dead zone), the panning operation may be suppressed. By suppressing the angle of view control in this manner, excessive driving can be prevented. In this case, it is expected that a tracking image with little discomfort, such as an image obtained by a person manually capturing a tracking image, can be obtained. Here, the tracking control parameter may indicate the size of the dead zone.

[0059] In the following example, the control amount of the pan operation and tilt operation is further determined according to the dead zone. As an example, FIG. 4 shows the relationship between the difference amount from the target position to the subject position and the control amount when the dead zone is set. When the difference amount is within the dead zone, the control amount is 0. When the difference amount is outside the dead zone, the control amount is determined according to the difference amount and the tracking sensitivity. In the case of FIG. 4, the control amount Y is determined as shown in formula (2). Here, W represents the size of the dead zone in the pan direction. Y = (XW) × K …… Equation (2)

[0060] FIG. 4 shows the control amount corresponding to each of a plurality of tracking sensitivities. Graph 140 shows the control amount when the tracking sensitivity is high. Graph 141 shows the control amount when the tracking sensitivity is medium. Graph 142 shows the control amount when the tracking sensitivity is low. In this way, the relationship between the difference amount and the control amount changes according to the tracking sensitivity. Similarly, the control amount of the tilt operation can also be calculated based on the tracking sensitivity and the magnitude of the difference in the tilt direction from the target position to the subject position. Note that the tracking sensitivity for calculating the control amount in the pan direction may be the same as or different from the tracking sensitivity for calculating the control amount in the tilt direction.

[0061] The relationship between the difference amount and the control amount is not limited to the above example. For example, the tracking sensitivity K may not be constant over the entire difference amount range. That is, the control amount may change linearly or nonlinearly depending on the difference amount.

[0062] The imaging control unit 134 can determine the control amount for one of the panning and tilting operations, or for both the panning and tilting operations, according to the tracking control parameters. The imaging control unit 134 may determine the control amount for the zooming operation in addition to or instead of at least one of the panning and tilting operations.

[0063] The zoom operation of the imaging device 200 can be controlled so that the subject appears larger in the image. The imaging control unit 134 can calculate the control amount required to bring the area of ​​the subject closer to the target area. For example, the imaging control unit 134 can calculate the control amount for the zoom operation based on the difference between the area of ​​the subject on the image and the target area of ​​the subject. Then, the imaging control unit 134 can calculate the control amount to eliminate this difference. The camera 201 performs a zoom operation (e.g., optical zoom or digital zoom) with a size according to such a control amount, so that the angle of view is controlled so that the area of ​​the subject approaches the target area. In this case, too, the imaging control unit 134 can determine the control amount for the zoom operation based on a tracking control parameter (e.g., tracking sensitivity). That is, even if the area of ​​the subject is constant, the imaging control unit 134 can determine the control amount so that the control amount becomes larger as the tracking sensitivity becomes larger. The imaging control unit 134 may also determine the control amount for the zoom operation based on a set dead zone.

[0064] So far, the case where the control amount represents the motion amount has been described. However, the control amount may represent the motion speed or motion acceleration. In this way, the imaging control unit 134 can control the change in the angle of view of the imaging device 200 based on the tracking control parameter (e.g., tracking sensitivity) in various ways. For example, when the tracking sensitivity is higher, a more rapid change in the angle of view of the imaging device 200 is allowed.

[0065] Figure 5 is a conceptual diagram showing object detection using a learning model. The learning model shown in Figure 5 is a multi-layered neural network whose internal parameters are adjusted by machine learning. Such a learning model can be called object detection AI.

[0066] The input data 150 corresponds to one frame image in a video captured by the imaging device 200. The input data 150 is an image obtained by the image processing unit 130 processing the frame image.

[0067] The output data 151 indicates the position of an object present in the input data 150. In the example of FIG. 5, the output data 151 indicates the tag, coordinates, and likelihood of the object. The tag indicates the name or type of the object. The tag is selected from tags included in the learning data used during learning. The coordinates identify the object area. In the example of FIG. 5, the coordinates include two points, coordinate 1 and coordinate 2. The coordinates 1 and coordinate 2 represent the top left point (coordinate 1) and the bottom right point (coordinate 2) of the circumscribing frame of the detected object as shown in the image 152. From the coordinates of these two points, the size and center coordinates of the object can be calculated. The likelihood indicates the reliability of the tag determined for the detected object. The likelihood may be a value between 0 and 1. A high likelihood value indicates a high reliability of the tag estimation result.

[0068] The learning model 153 is composed of a neural network. The internal parameters of the neural network are generated by a learning process by the learning unit 135. The learning process is performed using learning data. The learning data includes a set of input data and teacher data. The input data is an image, and the teacher data is a tag and coordinates (coordinate 1 and coordinate 2) related to an object appearing in the image. The learning unit 135 may include an error detection unit and an update unit. The error detection unit obtains an error between the teacher data and output data output from the output layer by inputting input data to the input layer of the neural network. The error detection unit may evaluate the error between the output data and the teacher data using a loss function. The update unit updates the internal parameters of the neural network based on the error obtained by the error detection unit. For example, the update unit can update the coupling weighting coefficient between the nodes of the neural network so as to reduce the error. The update unit can update the parameters using, for example, an error backpropagation method. The error backpropagation method is a method of adjusting the parameters of the neural network so as to reduce the above-mentioned error.

[0069] An overview of the operation of the automatic tracking photography system according to this embodiment will be described below. First, in step (1), the operation device 300 transmits an automatic photography start command to the information processing device 100. When a user operates the operation unit 302 of the operation device 300 to input an instruction to switch the operation of the imaging device 200 to automatic photography, the operation device 300 can transmit such a command.

[0070] In step (2), the information processing device 100 acquires an image captured by the imaging device 200. In step (3), the information processing device 100 acquires information indicating the current drive speed of the drive unit 202 from the imaging device 200. This drive speed can indicate the drive speeds of pan, tilt, and zoom.

[0071] In step (4), the information processing device 100 estimates the position of the subject in the acquired video. The information processing device 100 can determine whether the subject is captured according to a determined composition based on the current position of the subject. The information processing device 100 can also calculate the difference between a target position where the subject should be located according to the determined composition and the actual subject position. The information processing device 100 also stores the estimation result of the subject's position in the RAM 101. The past estimation result of the subject's position is used to calculate the subject's speed based on the change in the subject's position.

[0072] In step (5), the information processing device 100 acquires distribution information from the switching device 400. The distribution information is information indicating the video selected as the distribution video. In other words, the distribution information can indicate whether or not the video captured by a specific imaging device 200 is selected as the distribution video. Such distribution information is also called tally information. According to such distribution information, the information processing device 100 can know which of the multiple imaging devices 200 has captured the video that is being distributed. The information processing device 100 may also acquire the distribution video from the switching device 400. The information processing device 100 can link the acquired distribution information, the video acquired from the imaging device 200, and the position of the subject in the video, respectively.

[0073] In step (6), the information processing device 100 calculates the velocity of the subject. This velocity corresponds to the actual movement of the subject. The information processing device 100 can calculate the velocity of the subject based on the driving speed of the imaging device 200 and the position change of the subject in the video. The information processing device 100 can determine whether or not there has been a sudden movement of the subject based on the velocity of the subject.

[0074] In step (7), the information processing device 100 determines a tracking control parameter used when controlling the angle of view of the imaging device 200. In this example, the information processing device 100 determines a tracking sensitivity. The information processing device 100 can determine the tracking sensitivity based on the distributed information and the position of the subject. Furthermore, the information processing device 100 can determine a dead zone. The information processing device 100 can determine the dead zone based on the distributed information and the position of the subject. The information processing device 100 may further determine at least one of the tracking sensitivity and the dead zone based on the speed of the subject. A specific determination method will be described later with reference to FIG. 6.

[0075] In step (8), the information processing device 100 controls the imaging device 200 to track the subject according to the determined tracking control parameters. For example, the information processing device 100 can transmit information indicating the control amount for the pan operation and the tilt operation to the imaging device 200.

[0076] By repeating these steps (2) to (8), it is possible to automatically track and photograph the subject at a desired angle of view according to the distribution information and the position of the subject.

[0077] 6 is a flowchart showing details of a process for setting tracking control parameters according to this embodiment. In the following process, a process is performed for a specific imaging device 200. Hereinafter, this specific imaging device 200 is referred to as the imaging device to be processed.

[0078] First, in S701, the estimation unit 131 and the result processing unit 132 acquire position information indicating the position of a subject in a video captured by an imaging device to be processed. The position of the subject can be determined using an object detection AI. That is, the estimation unit 131 can estimate the position of the subject within the angle of view of the video using the learning model as described above.

[0079] In S702, the distribution information acquisition unit 137 acquires distribution information indicating the distribution status of the video captured by the imaging device to be processed, as described above. In S703, the imaging control unit 134 determines whether the video captured by the imaging device to be processed is a distributed video. If the video is a distributed video, the process proceeds to S704. If the video is not a distributed video, the process proceeds to S712.

[0080] The processing from S704 onwards is performed when the video captured by the processing target imaging device is being distributed. As described above, in this case, the imaging control unit 134 can set the tracking control parameters by the first method. Also, as described above, the imaging control unit 134 can set the composition by the first method. In the following example, the imaging control unit 134 sets the composition so that the subject is captured in the center of the angle of view.

[0081] Here, the imaging control unit 134 can set the tracking control parameters according to the position of the subject. Also, the imaging control unit 134 can further set the tracking control parameters according to the change in the position of the subject. In this example, the tracking control parameters are set according to the possibility that the subject will go out of the angle of view. For example, the imaging control unit 134 can set the tracking control parameters according to whether a first condition regarding the position of the subject is satisfied. In the example of FIG. 6, the first condition includes that the position of the subject is within a predetermined area of ​​the edge of the image.

[0082] Also, the imaging control unit 134 can set the tracking control parameters according to both the position of the subject and the position change of the subject. In the example of FIG. 6, the first condition further includes that the position change amount of the subject is equal to or greater than a threshold value. Specifically, in S704, the imaging control unit 134 can determine whether or not the first condition is satisfied that the position change of the subject is greater than a threshold value and the subject is at the edge of the angle of view. When the subject is within a predetermined distance from the edge of the image, the imaging control unit 134 can determine that the subject is at the edge of the angle of view. If the first condition is satisfied, the process proceeds to S705. If not, the process proceeds to S710.

[0083] The threshold value for the position change of the subject and the threshold value for the angle of view end can be set appropriately depending on the use case, tracking settings, etc. As an example, a use case in which a person is tracked and photographed in a lecture hall will be described. In this case, the subject is a person. Therefore, the speed and acceleration required for tracking are not high. Therefore, it is unlikely that the acceleration of the driving unit 202 of the imaging device 200 is not fast enough to track the subject. Therefore, in this case, the threshold value for the position change and the threshold value for the angle of view end can be set low. In this way, the imaging control unit 134 can set these threshold values ​​depending on the type of subject in the video.

[0084] Furthermore, the first condition is not limited to the above, and can be set so that the first condition is satisfied when there is a high possibility that the subject will move outside the angle of view.

[0085] S705 to S708 are performed when a video captured by the processing target imaging device is being distributed and a first condition regarding the position of the subject is satisfied. In this example, the tracking control parameters are set so that the subject can be captured according to a desired composition while suppressing stuttering in the video.

[0086] In S705, the imaging control unit 134 acquires information indicating the drive speed of the drive unit 202 of the imaging device 200. The imaging control unit 134 can acquire this information by communicating with the imaging device 200 as described above.

[0087] In S706, the imaging control unit 134 calculates the velocity of the subject. This velocity of the subject may be a value that depends on the driving speed of the imaging device 200 and the position change of the subject in the video. That is, the imaging control unit 134 can calculate the velocity of the subject based on the driving speed of the driving unit 202 and the position change of the subject in the video. The velocity of the subject corresponds to the velocity of the subject in real space. The velocity of the subject is obtained by removing the influence of the change in the angle of view of the imaging device 200 during imaging from the position change of the subject in the video. Note that the driving speed of the driving unit 202, the position change of the subject, and the velocity of the subject can be expressed based on the change in angle with respect to the optical center axis.

[0088] In S707, the imaging control unit 134 sets the tracking sensitivity. The imaging control unit 134 can set the tracking sensitivity according to the speed of the subject. For example, the imaging control unit 134 can set the tracking sensitivity so that the driving speed of the driving unit 202 is faster than the speed of the subject calculated in S706. In this way, by driving the driving unit 202 faster than the speed of the subject, the position of the subject changes significantly in the video, and the position of the subject becomes the target composition. For example, the imaging control unit 134 can set the tracking sensitivity K so that the driving speed is faster than the speed of the subject when the driving unit 202 drives according to the control amount determined according to formula (1) or (2) according to the current position of the subject. In this way, by taking the speed of the subject into consideration, a tracking sensitivity that is sufficiently fast to track the subject can be set. Note that the imaging control unit 134 may set the tracking sensitivity according to the position change of the subject in the video instead of the speed of the subject. In this way, in S707, the tracking control parameters can be set so that a faster angle of view control is performed compared to S710 described later.

[0089] During tracking shooting of a subject, the subject may move out of the angle of view due to a delay in tracking the subject that moves suddenly. In particular, when the driving speed is slow or the tracking sensitivity is low, it takes time to change the angle of view at a speed that matches the movement of the subject, so the subject is more likely to move out of the angle of view. In this embodiment, the tracking sensitivity can be increased only when there is a high possibility that the subject will move out of the angle of view. Stuttering in the video can be reduced. In particular, in this embodiment, even if the first condition is satisfied, it is possible to prevent the tracking sensitivity from being set too high by taking into account the speed of the subject.

[0090] In S708, the imaging control unit 134 determines the size of the dead zone. When the video captured by the imaging device to be processed is being distributed, the imaging control unit 134 can determine the size of the dead zone according to the tracking sensitivity. Specifically, when the tracking sensitivity is larger, the imaging control unit 134 can make the dead zone smaller.

[0091] For example, in S708, the imaging control unit 134 can increase the dead zone. The size of the dead zone may be, for example, 10% of the angle of view. The size of the dead zone can be set according to the use case and the tracking setting. As an example, a use case in which a person is tracked and photographed in a lecture hall is described. In this example, tracking photography is performed so that the person's face is captured at the target position. In such a use case, the person may make a movement such as tilting his / her head without moving. When tracking a person's face, panning and tilting operations of the imaging device may be performed in such a case. On the other hand, when a person operates a camera, the angle of view is often fixed in such a case. Therefore, by increasing the dead zone, it is possible to reduce the stuttering of the tracking image and reduce the discomfort in tracking photography. In this way, in S708, the tracking control parameters can be set so that a larger dead zone is provided compared to S711 described later.

[0092] As another example, the size of the dead zone set in S708 may be 20% of the size of the subject within the angle of view. In this case, the normal size of the dead zone may be 10% of the size of the subject. Also, the reduced size of the dead zone may be 5% of the size of the subject.

[0093] As described above, in S707 to S708, the imaging control unit 134 can set the tracking speed parameters (for example, the tracking sensitivity and the size of the dead zone) according to the speed of the subject. Also, in S705 to S708, the tracking sensitivity can be made larger than in S710 to S711 described later. Therefore, in S708, the dead zone can be set larger to reduce stuttering of the tracking image. In particular, when tracking shooting is performed using the tracking sensitivity set in S707, the angle of view is likely to change significantly. On the other hand, if the angle of view of the tracking image changes significantly, a sense of incongruity may increase. By making the dead zone larger in this way, excessive driving of the imaging device 200 is suppressed and a sense of incongruity is reduced when the position of the subject approaches the target position from the end of the angle of view.

[0094] S710 to S711 are performed when the image captured by the image capture device to be processed is being distributed and the first condition regarding the position of the subject is not satisfied. In this example, since the subject is close to the target position, the tracking control parameters are set so that the stuttering of the image is further suppressed.

[0095] In S710, the imaging control unit 134 sets a low tracking sensitivity. The tracking sensitivity set here may be a fixed value. The tracking sensitivity set here may be lower than the tracking sensitivity set in S707. By setting a low tracking sensitivity in this manner, it is possible to capture an image while smoothly tracking a moving subject. Therefore, the angle of view is prevented from being changed intermittently, and therefore stuttering in the tracking image can be suppressed.

[0096] In S711, the imaging control unit 134 determines the size of the dead zone. As described above, the imaging control unit 134 can determine the size of the dead zone depending on the tracking sensitivity. In this example, since a small tracking sensitivity is set, the imaging control unit 134 can make the dead zone smaller. For example, in S711, the imaging control unit 134 can make the dead zone smaller than the dead zone set in S708.

[0097] S712 to S715 are performed when the video captured by the imaging device to be processed is not being distributed. As described above, in this case, the imaging control unit 134 can set the tracking control parameters by the second method. In this example, since the video is not being distributed, the tracking video may have stuttering. For this reason, the tracking control parameters are set so that the subject can be maintained within the angle of view. On the other hand, as described above, the imaging control unit 134 can set the composition by the second method.

[0098] In S712, the imaging control unit 134 sets a composition according to the speed of the subject. The imaging device 200 that is not capturing the distribution video is often used as a backup when the subject in the distribution video falls outside the angle of view. For this reason, the imaging device 200 can be made to perform tracking shooting with a composition that is less likely to cause the subject to fall outside the angle of view. In this embodiment, when the speed of the subject is greater than a threshold, the imaging control unit 134 sets a front-opening composition. In the front-opening composition, the target position of the subject is offset from the center of the angle of view. Specifically, the target position of the subject can be offset in the opposite direction to the movement direction of the subject. On the other hand, when the speed of the subject is equal to or less than the threshold, the imaging control unit 134 sets a Japanese flag composition. In the Japanese flag composition, the target position of the subject is the center of the angle of view.

[0099] In S713, the imaging control unit 134 sets the tracking sensitivity. The imaging control unit 134 can set a high tracking sensitivity. That is, the imaging control unit 134 can set a tracking control parameter so that faster angle of view control is performed compared to the first method (S705 to S711). Specifically, the imaging control unit 134 can set the tracking sensitivity higher than the tracking sensitivity set in S707. Also, the imaging control unit 134 can set the tracking sensitivity higher than the tracking sensitivity set in S710. Since the video captured by the imaging device 200 is not immediately distributed, it is acceptable for this video to give an uncomfortable feeling. For this reason, a high tracking sensitivity can be set in order to prioritize preventing the subject from moving out of the angle of view.

[0100] In S714, the imaging control unit 134 sets the size of the dead zone. When the image captured by the imaging device to be processed is not being distributed, the imaging control unit 134 can reduce the dead zone. The image captured by the imaging device 200 is not immediately distributed. Therefore, in order to prioritize preventing the subject from moving outside the angle of view, the dead zone can be reduced even though a high tracking sensitivity is set. For example, in S714, the imaging control unit 134 can set the tracking control parameters so that a smaller dead zone is set than in S708. Also, the imaging control unit 134 can set the size of the dead zone regardless of the tracking sensitivity. For example, the size of the dead zone may be 5% of the size of the subject.

[0101] The imaging control unit 134 can control the imaging device 200 as described above to track the subject according to the tracking speed parameter thus set. In the above example, the imaging control unit 134 may set the tracking speed parameter further according to the type of subject in the video. For example, a high tracking sensitivity can be set for a fast-moving subject such as an airplane, compared to a slow-moving subject such as a person.

[0102] The configuration of the automatic tracking system is not limited to the configuration shown in FIG. 1 to FIG. 3. For example, in the above embodiment, the information processing device 100 determines the control amount according to the detection result of the subject, and the imaging device 200 controls the angle of view according to the transmitted control amount. However, the imaging device 200 may track the subject autonomously. That is, each imaging device 200 may detect the subject in the video. For this purpose, the imaging device 200 may have an estimation unit 131 and a result processing unit 132. Then, the imaging device 200 may determine the control amount according to the detection result of the subject, and control the angle of view according to the determined control amount. In this way, the imaging device 200 can control the angle of view so that the subject is captured at a position according to a predetermined composition. In this case, the imaging device 200 can determine the control amount as described above according to the tracking control parameter transmitted from the information processing device 100. Also, in such an embodiment, the information processing device 100 may acquire the detection result of the subject from the imaging device 200. The information processing device 100 can determine the tracking control parameter according to the detection result of the subject acquired from the imaging device 200. In such an embodiment, the information processing device 100 does not need to include the estimation unit 131 and the result processing unit 132.

[0103] In another embodiment, the imaging device 200 may not only autonomously track the subject but also determine the tracking control parameters. In this case, each imaging device 200 may have a functional unit similar to that of the information processing device 100. For example, the imaging device 200 and the information processing device 100 may be integrated.

[0104] Furthermore, the operation device 300 may include the information processing device 100. In this case, the imaging control device may have a RAM 101, a GPU 102, a CPU 103, an input unit 104, a storage unit 105, a communication unit 106, a UI unit 107, an operation unit 302, and a display unit 305. Furthermore, the operation device 300 may be connected to the information processing device 100 via a network 500.

[0105] In the above embodiment, the estimation unit 131 estimates the position of the subject using a machine-learned learning model. However, the method of estimating the position of the subject is not limited to this method. For example, the estimation unit 131 may estimate the position of the subject using rule-based processing. In one embodiment, the estimation unit 131 can use a look-up table (LUT). This LUT indicates the relationship between input data and output data. Such an LUT may be created in advance. Furthermore, the storage unit 105 of the information processing device 100 can store the created LUT. The estimation unit 131 can obtain output data by referring to the stored LUT. Furthermore, the LUT may be prepared as a program that performs the same processing as the estimation unit 131. Such an LUT can realize the processing of the estimation unit 131 by operating in cooperation with the CPU 103 or the GPU 102, etc.

[0106] In the above embodiment, the tracking sensitivity is set mainly based on the distribution information and the position of the subject. On the other hand, the imaging control unit 134 may set the size of the dead zone based on the distribution information and the position of the subject. For example, the imaging control unit 134 can reduce the dead zone when the image captured by the imaging device to be processed is being distributed and the position change of the subject is greater than a threshold value. Furthermore, the imaging control unit 134 can increase the dead zone when the image captured by the imaging device to be processed is being distributed and the position change of the subject is equal to or less than a threshold value. Furthermore, the imaging control unit 134 can reduce the dead zone when the image captured by the imaging device to be processed is not being distributed. In this way, the method of setting the tracking control parameters is not particularly limited.

[0107] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0108] The disclosure of this specification includes the following imaging control device, control method, and program. (Item 1) An imaging control device associated with an imaging device capable of controlling an angle of view so as to track an object according to a parameter indicating sensitivity of angle of view control to a position of the object, An acquisition means for acquiring distribution information indicating a distribution status of the video captured by the imaging device; a setting means for setting the parameters for the imaging device based on the distribution information and position information indicating a position of the subject in the video; An imaging control device comprising: (Item 2) 2. The imaging control device according to item 1, wherein the parameter is a parameter for controlling a speed or acceleration of angle of view control. (Item 3) When the object is included in a dead zone in the image, the angle of view of the imaging device is not changed, 3. The imaging control device according to claim 1, wherein the parameter indicates a size of the dead zone. (Item 4) The setting means is When the distribution information indicates that the video captured by the imaging device is being distributed, the parameters are set by a first method; The imaging control device according to any one of items 1 to 3, characterized in that when the distribution information indicates that the video captured by the imaging device is not being distributed, the parameters are set by a second method. (Item 5) In the first method, the setting means 5. The imaging control device according to item 4, wherein the parameters are set in accordance with both the position of the subject and a change in the position of the subject. (Item 6) In the first method, the setting means The imaging control device according to any one of items 4 to 5, characterized in that the parameters are set so that faster angle of view control is performed when a first condition regarding the position of the subject is satisfied, compared to when the first condition is not satisfied. (Item 7) 7. The imaging control device according to item 6, wherein the first condition includes that the position of the subject is within a predetermined area at an edge of the image and that an amount of change in the position of the subject is equal to or greater than a threshold value. (Item 8) In the first method, the setting means 8. The imaging control device according to any one of items 4 to 7, characterized in that, when a first condition regarding the position of the subject is satisfied, the parameter is set according to the speed of the subject. (Item 9) 8. The imaging control device according to item 7, wherein the velocity of the subject is a value that depends on a drive speed of the imaging device and a position change of the subject in the video. (Item 10) In the second method, the setting means 10. The imaging control device according to any one of items 4 to 9, characterized in that the parameters are set so that faster angle of view control is performed as compared to the first method. (Item 11) In the first method, the setting means setting the parameters such that a larger dead zone is provided when the first condition is satisfied compared to when the first condition is not satisfied; 11. The imaging control device according to any one of items 6 to 10, characterized in that, when the subject is included in the dead zone in the image, the angle of view of the imaging device is not changed. (Item 12) In the second method, the setting means setting the parameters so that a smaller dead zone is provided compared to a case where the first condition is satisfied in the first method; 12. The imaging control device according to any one of items 6 to 11, characterized in that, when the subject is included in the dead zone in the image, the angle of view of the imaging device is not changed. (Item 13) The setting means further sets a composition of the video based on the distribution information, 13. The imaging control device according to any one of items 1 to 12, characterized in that the composition indicates a target position of the subject in the image, and the angle of view of the imaging device is controlled according to the composition. (Item 14) Item 14. The imaging control device according to item 13, wherein the setting means further sets a composition of the image based on the speed of the subject. (Item 15) The imaging control device according to any one of items 1 to 14, characterized in that the setting means sets at least one of the parameters and the composition of the image further depending on the type of the subject detected from the image. (Item 16) the acquiring means acquires distribution information for each of a plurality of imaging devices; 16. The imaging control device according to any one of items 1 to 15, wherein the setting means sets the parameters for each of the plurality of imaging devices. (Item 17) 17. The imaging control device according to any one of items 1 to 16, characterized in that the setting means causes the imaging device to change the angle of view at a speed according to the parameter so that the position of the subject in the image approaches a target position. (Item 18) The imaging control device described in any one of items 1 to 17, characterized in that the distribution information is information indicating an image selected as the image to be distributed from among images captured by the multiple imaging devices. (Item 19) A control method performed by an imaging control device associated with an imaging device capable of controlling an angle of view so as to track an object according to a parameter indicating sensitivity of angle of view control to a position of the object, the method comprising: acquiring distribution information indicating a distribution status of the video captured by the imaging device; setting the parameters for the imaging device based on the distribution information and position information indicating a position of the subject in the video; A control method comprising the steps of: (Item 20) A program for causing a computer to function as the imaging control device according to any one of items 1 to 18.

[0109] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0110] 100: information processing device, 130: image processing unit, 131: estimation unit, 132: result processing unit, 133: movement prediction unit, 134: imaging control unit, 135: learning unit, 136: recording unit, 137: distribution information acquisition unit

Claims

1. An imaging control device associated with an imaging device capable of controlling an angle of view so as to track an object according to a parameter indicating sensitivity of angle of view control to a position of the object, An acquisition means for acquiring distribution information indicating a distribution status of the video captured by the imaging device; a setting means for setting the parameters for the imaging device based on the distribution information and position information indicating a position of the subject in the video; An imaging control device comprising:

2. 2. The imaging control device according to claim 1, wherein the parameter is a parameter for controlling a speed or an acceleration of an angle of view control.

3. When the object is included in a dead zone in the image, the angle of view of the imaging device is not changed, 2. The imaging control device according to claim 1, wherein the parameter indicates a size of the dead zone.

4. The setting means is When the distribution information indicates that the video captured by the imaging device is being distributed, the parameter is set by a first method; The imaging control device according to claim 1 , wherein, when the distribution information indicates that the video captured by the imaging device is not being distributed, the parameter is set by a second method.

5. In the first method, the setting means 5. The imaging control device according to claim 4, wherein the parameter is set in accordance with both the position of the subject and a change in the position of the subject.

6. In the first method, the setting means 5. The imaging control device according to claim 4, wherein the parameters are set so that, when a first condition regarding the position of the subject is satisfied, faster angle of view control is performed compared to when the first condition is not satisfied.

7. 7. The imaging control device according to claim 6, wherein the first condition includes that the position of the subject is within a predetermined area at an edge of the image, and an amount of change in position of the subject is equal to or greater than a threshold value.

8. In the first method, the setting means 5. The imaging control device according to claim 4, wherein the parameter is set in accordance with the speed of the subject when a first condition regarding the position of the subject is satisfied.

9. 8. The imaging control device according to claim 7, wherein the velocity of the subject is a value that depends on a drive speed of the imaging device and a position change of the subject in the video.

10. In the second method, the setting means 5. The imaging control device according to claim 4, wherein the parameters are set so that the angle of view can be controlled at a higher speed than in the first method.

11. In the first method, the setting means setting the parameters such that a larger dead zone is provided when the first condition is satisfied compared to when the first condition is not satisfied; The imaging control device according to claim 6 , wherein, when the subject is included in the dead zone in the image, the angle of view of the imaging device is not changed.

12. In the second method, the setting means setting the parameters so that a smaller dead zone is provided compared to a case where the first condition is satisfied in the first method; The imaging control device according to claim 6 , wherein, when the subject is included in the dead zone in the image, the angle of view of the imaging device is not changed.

13. The setting means further sets a composition of the video based on the distribution information, 2. The imaging control device according to claim 1, wherein the composition indicates a target position of the subject in the image, and the angle of view of the imaging device is controlled in accordance with the composition.

14. 14. The imaging control device according to claim 13, wherein the setting means sets a composition of the image further based on a speed of the subject.

15. 2. The imaging control device according to claim 1, wherein the setting means sets at least one of the parameters and a composition of the image further depending on the type of the subject detected from the image.

16. the acquiring means acquires distribution information for each of a plurality of imaging devices; 2. The imaging control device according to claim 1, wherein the setting means sets the parameters for each of the plurality of imaging devices.

17. 2. The imaging control device according to claim 1, wherein said setting means causes said imaging device to change the angle of view at a speed according to said parameter so that the position of said subject in said video approaches a target position.

18. The imaging control device according to claim 1 , wherein the distribution information is information indicating an image selected as an image to be distributed from among images captured by the plurality of imaging devices.

19. A control method performed by an imaging control device associated with an imaging device capable of controlling an angle of view so as to track an object according to a parameter indicating sensitivity of angle of view control to a position of the object, the method comprising: acquiring distribution information indicating a distribution status of the video captured by the imaging device; setting the parameters for the imaging device based on the distribution information and position information indicating a position of the subject in the video; A control method comprising the steps of:

20. A program for causing a computer to function as the imaging control device according to any one of claims 1 to 18.

Images