Information processing device, control method, and program

Abstract

To improve the convenience of shooting based on pre-configured settings. [Solution] The information processing device acquires information indicating multiple setting values ​​that set the field of view, which correspond to multiple positions included in a predetermined path when taking pictures while moving the field of view of the imaging device along that predetermined path. Based on two of the multiple setting values, it sets a target value for setting the field of view that is different from the multiple setting values, and changes the settings of the imaging device toward the target value.

Description

[Technical Field]

[0001] This disclosure relates to control technology for imaging devices. [Background technology]

[0002] In recent years, with the expansion of the video distribution market, there has been a need to remotely control the pan, tilt, and zoom of an imaging device using a network to track and capture subjects. Conventional tracking shooting methods include an automatic loop function that uses a preset function to save parameters for when shooting a point of interest, pre-specifying parameters for multiple designated locations, and then circulating and shooting at those locations. There is also a trace function that allows the user to control the imaging device by pre-recording multiple sets of parameters corresponding to the imaging range and image quality of the imaging device, and then referencing these parameter sets to perform shooting, thereby tracing the user's control. For example, when shooting subjects that move along a predetermined path, such as in track and field events, the above functions allow the imaging device to be controlled according to pre-recorded parameters. Patent Document 1 discloses a method for editing recorded parameters for tracing. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-155733 [Overview of the project] [Problems that the invention aims to solve]

[0004] For example, when photographing track and field events, if the speed at which the subjects move differs from what was expected, such as the tactics between competitors, the shooting based on pre-recorded parameters may not function properly, potentially impairing convenience. [Means for solving the problem]

[0005] The present disclosure provides a technique for improving the convenience of shooting based on preset settings.

[0006] An information processing apparatus according to an aspect of the present disclosure includes acquisition means for acquiring information indicating a plurality of setting values for setting the shooting angle, the plurality of setting values corresponding to a plurality of positions included in a predetermined path when shooting while moving the shooting angle of an imaging device along the predetermined path; setting means for setting a target value for setting the shooting angle, different from the plurality of setting values, based on two setting values included in the plurality of setting values; and changing means for changing the setting of the imaging device toward the target value.

Advantages of the Invention

[0007] According to the present disclosure, the convenience of shooting based on preset settings can be improved.

Brief Description of the Drawings

[0008] [Figure 1] It is a diagram showing an example of a system configuration and preset information. [Figure 2] It is a diagram showing an example of the hardware configuration of an information processing apparatus. [Figure 3] It is a diagram showing an example of the functional configuration of an information processing apparatus. [Figure 4] It is a diagram for explaining an example of preset information and target setting of an imaging device. [Figure 5] It is a diagram for explaining an example of target update. [Figure 6] It is a diagram showing an example of the configuration of a controller. [Figure 7] It is a diagram showing an example of the flow of processing executed by an information processing apparatus.

Modes for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] (System Configuration) FIG. 1(A) shows a configuration example of an imaging system according to the present embodiment. This imaging system includes an imaging device 101, an information processing device 102, and a controller 103. The imaging device 101 has a function of tracking and imaging subjects 112 to 113 such as athletes within a predetermined range, such as a track 111 for land sports, by changing at least one of the pan value, tilt value, and zoom value (PTZ value) over time. In FIG. 1(A), only one imaging device 101 is shown, but a plurality of imaging devices 101 may be prepared. The information processing device 102 acquires information indicating the current state from the imaging device 101, and also acquires information indicating a user operation from the controller 103, and performs information processing for controlling the shooting range by the imaging device 101, for example. The controller 103 is a device used by the user to control the imaging device 101. When there are a plurality of imaging devices 101, a plurality of information processing devices 102 and controllers 103 corresponding to each of the plurality of imaging devices 101 may be prepared. However, this is just an example, and a set of information processing device 102 and controller 103 for collectively handling the plurality of imaging devices 101 may be prepared.

[0011] In this embodiment, the information processing device 102 controls the imaging device 101 to image subjects 112-113 while moving the field of view along the track based on pre-set preset information. The preset information is information that associates a specific position with a PTZ value, for example, as shown in positions 121-128 on the track. In other words, the preset information is information that sets what composition the imaging device 101 will use to capture images at a specific position within the target. Figure 1(B) shows an example of this preset information. Each position corresponding to the preset information is assigned identification information (ID), and each ID is associated with a combination of setting information for pan value (Pan), tilt value (Tilt), and zoom value (Zoom). For example, position 121 corresponds to ID=1, position 122 corresponds to ID=2, position 123 corresponds to ID=3, ..., position 128 corresponds to ID=8. Pan and Tilt indicate the pan and tilt rotation angles of the imaging device 101, i.e., the orientation of the imaging device 101. Zoom represents the horizontal field of view of the imaging device 101. In Figure 1(B), the units for pan, tilt, and zoom values ​​are "millidegrees," for example, a value of 10000 represents "10°." In one example, by specifying ID=1 as the start ID and ID=8 as the end ID, the imaging device 101 obtains PTZ values ​​corresponding to ID=1 through ID=8. While specifying only the start and end IDs may indicate that the ID settings between the start and end IDs are used sequentially, incrementing by one ID each time, this is not the only possible interpretation. For example, the imaging device 101 may obtain information indicating the order of the settings used, such as start ID, second ID, third ID, ..., and then obtain PTZ values ​​corresponding to the IDs included in that order.

[0012] The imaging device 101 then performs imaging by switching PTZ values ​​so that the PTZ value of the next ID (e.g., ID=2) is used after a predetermined time has elapsed since the start of imaging using one PTZ value (e.g., ID=1). The predetermined time may be included in the setting information, or it may be entered separately by user operation. Furthermore, the imaging device 101 may use an intermediate PTZ value so that it gradually changes from the PTZ value corresponding to the first ID to the PTZ value corresponding to the second ID between the start of imaging using the PTZ value corresponding to the first ID and the timing corresponding to the next second ID. In other words, an interpolated value of the PTZ value may be calculated between the two preset values. For example, when half the time has elapsed from the start of using the PTZ value of the first ID to the start of the transition to the second ID, a value near the midpoint between the PTZ value of the first ID and the PTZ value of the second ID may be used as the PTZ value. The imaging device 101 can move the field of view at each timing based on preset PTZ settings as shown in Figure 1(B) to track subjects 112-113 and take images.

[0013] In this embodiment, the information processing device 102 further changes the field of view captured by the imaging device 101 based on user operation via the controller 103. This adjusts the field of view captured by the imaging device 101 in accordance with changes in the moving speed of the subjects 112-113, ensuring that the subjects 112-113 are properly tracked and photographed. In other words, the information processing device 102 receives user operation via the controller 103 and controls the imaging device 101 to track and photograph the subjects 112-113 even when the speed at which the subjects 112-113 move around the track changes.

[0014] In track and field events, the speed at which the subjects (subjects 112-113) circle the track changes due to the tactics and strategies employed by the competitors. Therefore, in order to properly photograph subjects 112-113, it is necessary to change the field of view movement speed of the imaging device 101 to match the movement speed of subjects 112-113. However, with the circulating shooting function using preset information as described above, it is not possible to dynamically change the field of view movement speed in accordance with the movement of the subjects. Furthermore, when the field of view moves near the position corresponding to the specified preset information, its movement speed decreases, and it stops once it reaches that position. As a result, it is not possible to smoothly track and photograph the subjects.

[0015] In this embodiment, in view of these circumstances, a technology is provided that enables smooth tracking and shooting of subjects 112-113 while utilizing automatic shooting using preset information. In this embodiment, a control value (control value) is used separately from the setting value so that the movement of the field of view is not slowed down when the difference between the value indicating the state of the imaging device 101 (PTZ value) and the setting value (PTZ value) specified by the preset information falls below a predetermined value. For example, when focusing on two adjacent positions along a path among a plurality of field of view positions that define the shooting path, a target position is set behind the second position corresponding to the second setting value, which is used later, as seen from the first position corresponding to the first setting value, which is used first. The setting value corresponding to that target position is then set as the control value described above. For example, the first setting value and the second setting value may be expressed as coordinates, and the control value may be prepared such that the second setting value exists between the control value and the first setting value on the straight line connecting the first and second setting values ​​in that coordinate space. The straight line described above may, in some cases, be a predetermined curve obtained by a predetermined interpolation process such as spline interpolation. By setting the control value as the target when changing the PTZ value of the imaging device 101 toward the second set value, it is possible to prevent the rate of change of the PTZ value from decreasing before reaching the second set value. In addition, when the difference between the current PTZ value of the imaging device 101 and the second set value falls below a predetermined value (i.e., before approaching the control value), the set value is updated to the third set value following the second set value. Alternatively, the set value may be updated when the difference between the position corresponding to the second set value and the current field of view position falls below a predetermined value. This prevents the current PTZ value of the imaging device 101 from passing the set value and approaching the control value. Furthermore, these techniques allow the user to flexibly track subjects 112-113 in accordance with their movement speed. That is, since the set value is updated to the next preset third set value based on the difference between the current PTZ value of the imaging device 101 and the second set value, imaging can continue along a pre-set path even if the rate of change of the PTZ value is changed.

[0016] The configuration of the information processing device 102 for performing such processing and the flow of the processing to be performed will be described below. In this embodiment, an example is shown in which the information processing device 102 exists separately from the imaging device 101, but for example, the information processing device 102 may be installed inside the imaging device 101. In this case, a remote controller 103 can transmit operation information to the information processing device 102 inside the imaging device 101, for example via a network, and the information processing device 102 can execute the processing described later based on that operation information.

[0017] (Configuration of the information processing device 102) Figure 2 shows an example of the hardware configuration of the information processing device 102. The information processing device 102 includes, for example, a CPU 201, ROM 202, RAM 203, auxiliary storage device 204, display unit 205, operation unit 206, communication I / F 207, and bus 208. Note that CPU stands for Central Processing Unit, ROM for Read Only Memory, RAM for Random Access Memory, and I / F for Interface.

[0018] The CPU 201 controls the entire device using computer programs and data stored in the ROM 202 and RAM 203, thereby realizing each function of the information processing device 102. Note that the CPU 201 is an example of one or more processors and may be replaced by other processors such as an MPU (Micro Processing Unit). Furthermore, dedicated hardware such as application-specific integrated circuits (ASICs), digital signal processors (DSPs), and field-programmable gate arrays (FPGAs) may be used in place of or in addition to the CPU 201. Such dedicated hardware can perform the processing described later, either in cooperation with the CPU 201 or independently. The ROM 202 stores programs that do not require modification, and the RAM 203 temporarily stores programs and data supplied from the auxiliary storage device 204, as well as data supplied externally via the communication I / F 207. In this embodiment, the RAM 203 stores tracking state information representing the control state of the imaging device 101. ROM 202 and RAM 203 are examples of one or more memories for the information processing device 102 to permanently or temporarily hold information, and internal storage devices with similar functions may be used. The auxiliary storage device 204 is a storage device such as a hard disk drive (HDD) or solid-state drive (SSD), and stores various data such as image data and audio data. The display unit 205 is composed of a device that presents information to the user visually, such as a liquid crystal display or a light-emitting diode (LED). The display unit 205 displays, for example, a Graphical User Interface (GUI) for the user to operate the information processing device 102. The display unit 205 may also have a function to present information by means of sound or vibration, for example. The operation unit 206 is an interface for receiving operations from the user. The operation unit 206 may include, for example, an interface connected to the controller 103, and may be configured to acquire information indicating user operations to the controller 103 from the controller 103 and transfer that information to the CPU 201.Furthermore, the control unit 206 and the communication interface 207 communicate with an external network, such as the Internet. The communication interface 207 may also be used to connect to the controller 103 via the network. The bus 208 interconnects the CPU 101 to the communication interface 207. Note that "bus" is just an example, and the interconnection may be established by other configurations.

[0019] Figure 3 shows an example of the functional configuration of the information processing device 102. The information processing device 102 includes a camera information acquisition unit 301, a first target group creation unit 302, a second target group creation unit 303, a target update unit 304, an operation information acquisition unit 305, a parameter creation unit 306, and an imaging device control unit 307. These functional units can be implemented, for example, by the CPU 201 executing programs stored in the ROM 202 or auxiliary storage device 204. Alternatively, at least one of these functional units may be implemented by dedicated hardware.

[0020] The camera information acquisition unit 301 acquires camera information transmitted from the imaging device 101. The camera information may include, for example, identification information to identify the imaging device 101, date and time information, the current PTZ value, and preset information which is a previously recorded point of interest. For example, the camera information acquisition unit 301 acquires preset information during the initialization process and acquires the current PTZ value during the steady-state processing after shooting has started. The preset information is recorded in advance, for example, along the course of a track, as described above using Figure 1(B). During the initialization process, the camera information acquisition unit 301 accepts user operations via a UI (User Interface) screen (not shown) and acquires arbitrary preset information.

[0021] The first target group creation unit 302 creates a first target group consisting of one or more first targets related to PTZ values, based on the preset information acquired by the camera information acquisition unit 301 during the initialization process. The first target is treated as information indicating the point that the field of view should pass through when the imaging device 101 is shooting to track the subject, and is expressed by a PTZ value corresponding to a specific point. Therefore, the first target corresponds to a setting value for controlling the field of view of the imaging device 101. The first target may also be a PTZ value indicated in the preset. The first target group is created so that the position where the image is taken by the PTZ value corresponding to the start ID among the acquired preset information is the starting point, and the position where the image is taken by the PTZ value corresponding to the end ID among the same preset information is the goal point. Then, by controlling the imaging device 101 to apply the first targets in order from the starting point to the goal point, the field of view can be moved along a pre-set path. In one example, the system may be set so that the field of view moves to the position corresponding to the start ID after passing the position corresponding to the end ID. For example, in the examples in Figures 1(A) and 1(B), position 121, which is captured with a PTZ value of ID=1, is the starting point, and position 128, which is captured with a PTZ value of ID=8, is the goal point. This preset information indicates that the field of view will move from position 121, passing through positions 122, 123, ..., 127, which are captured with PTZ values ​​corresponding to IDs 2, 3, ..., 7, respectively, to reach position 128. The first target group creation unit 302 may perform interpolation between two PTZ values ​​included in the preset information and create one or more interpolated values ​​as the first target. For example, spline interpolation can be used for the interpolation process here. By creating the first target group through interpolation, it is possible to interpolate smoothly between positions corresponding to the preset information when only a small amount of preset information is prepared in advance. This makes it possible to reduce the number of preset information records that need to be recorded in advance.

[0022] The second target group creation unit 303 creates a second target group consisting of second targets corresponding to the first targets. The second targets are treated as targets for controlling the position captured by the imaging device 101 so that it passes through the position corresponding to the first target. For example, in order to pass through the position corresponding to the i-th first target, the second target group creation unit 303 sets the i-th second target on the extension of the line from the position corresponding to the i-th first target to the position corresponding to the i-th first target. The second targets correspond to the set value for controlling the field of view of the imaging device 101.

[0023] Here, we will explain how to create the second target using Figure 4. Figure 4 shows a simplified coordinate system with the horizontal axis representing the pan value and the vertical axis representing the tilt value. Since the position captured by the imaging device 101 corresponds to the PT value (pan and tilt value), the coordinates representing the PT value correspond one-to-one with the coordinates of the position being captured. Therefore, by changing the field of view of the imaging device 101 toward each target value, the imaging device 101 can capture the set position along the set path. In Figure 4, a vector 411 is shown moving from coordinate 401 corresponding to the i-1th first target toward coordinate 402 corresponding to the i-th first target. At this time, the second target 403 is set at a position further by an arbitrary distance value 421 in the direction of vector 411 from coordinate 402 indicating the first target. This distance value 421 can be set in advance by accepting user input, for example, through the controller 103 or a UI screen (not shown). Note that this is just one example, and a predetermined constant value in the system may be used as the distance value 421. The second target is calculated from the first target, as shown in equation (1) below.

number

number

[0024] The target update unit 304 acquires camera information from the camera information acquisition unit 301 and acquires the first target group and the second target group from the first target group creation unit 302 and the second target group creation unit 303, respectively. The target update unit 304 then determines whether the position of the field of view corresponding to the current PTZ value of the imaging device 101, which has been acquired as camera information, has passed the position corresponding to the currently set (for example, the i-1th) first target. This determination can be made, for example, by comparing the distance between a first coordinate system, which has the current pan value and tilt value of the imaging device 101 as elements, and a second coordinate system, which has the pan value and tilt value of the currently set first target as elements, with a predetermined value. Alternatively, this determination may be made by comparing the distance between the center position of the imaged field of view and the position corresponding to the first target with a predetermined value. The target update unit 304 can determine that the position of the field of view corresponding to the current PTZ value of the imaging device 101 has passed the position corresponding to the currently set first target when the aforementioned distance falls below a predetermined value. If the target update unit 304 determines that the position of the field of view corresponding to the current PTZ value of the imaging device 101 has passed the position corresponding to the currently set first target, it performs an update process to use the next (for example, the i-th) first target. The target update unit 304 also updates the second target based on the updated first target. The first target corresponding to the starting point may be set as the initial value of the first target, and the second target calculated based on the first target and a provisional target value corresponding to a predetermined position may be set as the initial value of the second target. The provisional target value may be, for example, the initial value of the PTZ value of the imaging device 101. In other words, the initial value of the second target can be calculated using the vector from the initial PTZ value of the imaging device 101 toward the first target and a predetermined distance value, as shown in equation (1) above. The provisional target value may also be a predetermined value that has been set in advance, such as the origin in the coordinate system. Furthermore, the first value of the first target group may be used as the starting position for imaging of the imaging device 101, and the initial value of the first target may be the second value of the first target group. In this case, the initial value of the second target can be calculated based on the first and second values ​​of the first target group, as shown in equation (1).

[0025] The update of the target will be explained using Figure 5. Figure 5 plots the pan and tilt values ​​501 acquired at a certain time t against the simplified coordinate system of Figure 4. At a certain time t, the PTZ value of the imaging device 101 is controlled to pass the coordinate 402 indicating the first target with the coordinate 403 indicating the second target as the target, after passing the coordinate 401 indicating the previous first target. The target update unit 304 updates the first and second targets in response to the distance between the coordinate 501 indicating the pan and tilt values ​​and the coordinate 402 indicating the first target becoming less than or equal to a predetermined value set in advance. For example, the target update unit 304 updates the first target to the value indicated by coordinate 502 and updates the second target to the value indicated by coordinate 503. In this way, in this embodiment, the first and second targets are updated regardless of the passage of time, in response to the PTZ value of the imaging device 101 approaching the first target for which it has been set. This makes it possible to cycle and take pictures at positions appropriately identified by preset information, regardless of the PTZ value movement speed. Therefore, it is possible to take pictures while appropriately tracking subjects 112-113 using the user's control of the PTZ value movement speed, without deviating from the shooting path identified by the preset information.

[0026] The operation information acquisition unit 305 acquires operation information input from the controller 103 and notifies the parameter creation unit 306 and the imaging device control unit 307 of this operation information. The parameter creation unit 306 first acquires a value indicating the second target determined by the target update unit 304. Next, the parameter creation unit 306 acquires a speed value from the operation information input from the operation information acquisition unit 305 and calculates the control speed for each PTZ value. Finally, the parameter creation unit 306 determines the control parameters for controlling the movement of the imaging device 101's PTZ value toward the second target at the calculated control speed. The imaging device control unit 307 acquires the control parameters created by the parameter creation unit 306 and transmits a control signal to control the imaging device 101. The imaging device control unit 307 also transmits a control signal to control the imaging device 101 based on the operation information input by the operation information acquisition unit 305. Furthermore, at least a portion of the information corresponding to the operation information acquired by the operation information acquisition unit 305 may be stored in a storage device such as RAM 203, and the parameter creation unit 306 and the imaging device control unit 307 may refer to RAM 203 or the like to acquire that information.

[0027] Here, an example of the controller 103 in this embodiment and its operation will be explained using Figure 6. In this embodiment, a general gamepad as shown in Figure 6 is used as the controller 103. In one example, the controller 103 receives operation information from the operation information acquisition unit 305 by receiving operation via buttons 601-605 and buttons 611-614. The operation information acquisition unit 305 transmits the acquired operation information to the parameter creation unit 306 and the imaging device control unit 307. As a result, the user can control the imaging device 101 by operating the controller 103.

[0028] The following describes an example of the relationship between the operation of each button on the controller 103 and the control of the imaging device 101. In this example, the user controls the pan and tilt speed of the imaging device 101 according to the degree to which the button 601 is pressed. For example, the state where the button 601 is not pressed is represented as 0.0, and the state where it is fully pressed is represented as 1.0. The controller 103 notifies the operation information acquisition unit 305 of a value between 0.0 and 1.0 as operation information, according to the degree of pressing. The operation information acquisition unit 305 supplies the acquired value as speed information to the parameter creation unit 306. The parameter creation unit 306 determines the control speed of the pan and tilt values ​​based on this speed information.

[0029] Furthermore, the user can instruct the information processing device 102 to move the field of view captured by the imaging device 101 back to its initial position by pressing button 602. Here, the initial position may be, for example, the position corresponding to the first value in the first target group, or a predetermined position such as the position corresponding to the initial orientation of the imaging device 101. When the operation information acquisition unit 305 acquires this operation information, it transfers the information to the imaging device control unit 307, and the imaging device control unit 307 controls the imaging device 101 to return the field of view to its initial state. In one example, when the operation information acquisition unit 305 receives this operation information, it transfers this information to the target update unit 304, and the target update unit 304 may return the first target and the second target to their initial values. The target update unit 304 may also return the first target and the second target to their initial values ​​in response to the fact that the field of view of the imaging device 101 has returned to its initial value based on the camera information of the imaging device 101.

[0030] Furthermore, the user can instruct the information processing device 102 to start or resume tracking shooting by pressing button 603. When the operation information acquisition unit 305 acquires this operation information, it may store information indicating the tracking state, which is set to "Tracking," in, for example, RAM 203. The imaging device control unit 307 refers to RAM 203 and transmits the control parameters acquired from the parameter creation unit 306 to the imaging device 101 while the information indicating the tracking state is set to "Tracking." The operation information acquisition unit 305 may also directly notify the imaging device control unit 307 of the information instructing the start or resumption of tracking shooting. This allows for control restrictions to be imposed so that tracking shooting is only possible when instructed by the user. Additionally, while the information indicating the tracking state is set to "Tracking," restrictions may be imposed such as not transmitting operation information from, for example, the operation of button 602. This prevents the angle of view captured by the imaging device 101 from moving to its initial position during tracking shooting.

[0031] Furthermore, the user can instruct the information processing device 102 to temporarily pause tracking shooting by pressing button 604. Upon acquiring this operation information, the operation information acquisition unit 305 may store information indicating the tracking state set to "paused" in, for example, RAM 203. Upon referring to this information, the imaging device control unit 307 stops tracking shooting by sending a control signal to the imaging device 101 to stop PTZ control. Alternatively, the operation information acquisition unit 305 may directly notify the imaging device control unit 307 of the information instructing the temporary pause of tracking shooting.

[0032] Furthermore, the user can instruct the information processing device 102 to end tracking shooting by pressing button 605. Upon acquiring this operation information, the operation information acquisition unit 305 stores information indicating the tracking state set to "end" in the RAM 203. The imaging device control unit 307 then terminates tracking shooting by sending a control signal to the imaging device 101 to terminate PTZ control.

[0033] Furthermore, the user can instruct the information processing device 102 to control the zoom in the telephoto direction while button 611 is pressed, and in the wide-angle direction while button 612 is pressed. The operation information acquisition unit 305 acquires operation information indicating whether buttons 611 and 612 are pressed (such as the duration of the press), and transmits this operation information to the parameter creation unit 306. The parameter creation unit 306 creates parameters to control the zoom based on this information. This makes it possible, for example, to take close-ups of a specific subject or to take pictures so that the entire subject fits within the field of view when tracking multiple subjects. Note that the zoom may not be operated by the user. Also, for example, the zoom function may be turned on or off using buttons 613 or 614. Note that the speed at which the Z value of the imaging device 101 changes in the telephoto direction (or wide-angle direction) may be controlled by the duration of the press of button 611 (or button 612). For example, the longer the button is pressed, the faster the Z-value of the imaging device 101 may be changed. Alternatively, the zoom of the imaging device 101 may be controlled to the telephoto direction (or wide-angle direction) not while the button 611 (or button 612) is pressed, but as a result of pressing the button. Furthermore, the speed at which the Z-value of the imaging device 101 is changed may be controlled by the number of times the button is pressed. For example, if button 611 is pressed once, the zoom of the imaging device 101 may be changed to the telephoto direction at a predetermined speed. If button 611 is pressed again in that state, the zoom of the imaging device 101 may be changed to the telephoto direction at a faster speed than the predetermined speed for a single press. If button 611 is pressed once more in that state, the zoom of the imaging device 101 may be changed to the telephoto direction even faster than before. On the other hand, if button 612 is pressed once in that state, the zoom of the imaging device 101 may be changed to the telephoto direction at the same speed as when button 611 is pressed twice. In other words, the zoom may be changed to the telephoto or wide-angle direction, and the speed of the change may be controlled by the number of times each of buttons 611 and 612 is pressed.Furthermore, to simplify user operation, buttons 611 and 612 may each be used to control the zoom change speed. For example, button 611 may be used to increase the zoom change speed, and button 612 may be used to decrease the zoom change speed. In this case, the target zoom value may be set to one of the following: the first target zoom value, the second target zoom value, or the zoom value corresponding to the next preset ID.

[0034] (Process flow) Next, using Figure 7, we will explain the process of controlling the imaging device 101, which is executed by the information processing device 102. In the following, the functional units shown in Figure 3 are specified as the main entities executing the operations, but since these functional units can be implemented using the CPU 101 or dedicated hardware, it may be interpreted that each processing step is executed by the CPU 101 or the like.

[0035] In S701, the camera information acquisition unit 301 performs an initialization process to acquire preset information corresponding to an ID specified by, for example, an ID acquired from the imaging device 101 or a user operation. Then, in S702, the first target group creation unit 302 uses the preset information acquired in S701 to create a first target group by, for example, performing spline interpolation. In S703, the second target group creation unit 303 creates a second target group from the first target group created in S702.

[0036] Then, in S704, the imaging device control unit 307 controls the imaging device 101 so that its PTZ value is moved to a value corresponding to the starting point of the first target group created in S702. At this time, the first and second targets used for tracking imaging are also set to their initial values.

[0037] In S705, the operation information acquisition unit 305 acquires operation information of buttons 601 to 605 from the controller 103. At this time, the operation information acquisition unit 305 stores information indicating the tracking state set to "tracking", "pause", or "end" in the RAM 203 according to the operation information. In S706, for example, the CPU 101 checks the information indicating the tracking state stored in the RAM 203 and switches the process according to the check result. When the information indicating the tracking state is "pause", the CPU 101 returns the process to S705 and waits until an operation for setting the information indicating the tracking state to "tracking" by the operation information acquisition unit 305 is accepted. When the information indicating the tracking state is "end", the imaging device control unit 307 performs a process for ending the tracking shooting on the imaging device 101. When the information indicating the tracking state is "tracking", the CPU 101 advances the process to S707.

[0038] In S707, the camera information acquisition unit 301 acquires camera information (current PTZ value) from the imaging device 101. Note that the camera information acquisition unit 301 can acquire this information periodically. Also, the camera information acquisition unit 301 may acquire this information even when not in the tracking state, or may acquire this information only during the tracking. Then, in S708, the target update unit 304 determines whether the position of the viewing angle corresponding to the current PTZ value acquired in S707 has passed the position corresponding to the currently set first target. Note that this determination may be made only by comparing the PTZ values regardless of the geographical position. That is, the determination may be made based on whether the distance between the current PTZ value and the PTZ value indicated by the first target becomes less than or equal to a predetermined value. This distance is, for example, {(Pan t -Pan c ) 2 +(Tilt t -Tilt c ) 2 +(Zoom t -Zoom c ) 2} 1 / 2It can be calculated as follows. Note that Pan is the pan value, Tilt is the tilt value, and Zoom is the zoom value, with the subscript t corresponding to the PTZ value of the first target and the subscript c corresponding to the current PTZ value. If the target update unit 304 determines that the position of the field of view corresponding to the current PTZ value has passed the position corresponding to the currently set first target, it updates the first and second targets in S709. On the other hand, if the target update unit 304 determines that the position of the field of view corresponding to the current PTZ value has not passed the position corresponding to the currently set first target, it proceeds to S710 without updating the first and second targets.

[0039] In S710, the operation information acquisition unit 305 performs a process to convert the operation information acquired in S705 into a speed value. If the range of the PTZ value of the imaging device 101 is 0 to 100, the operation information acquisition unit 305 maps the operation information in the range of 0.0 to 1.0 to a speed value of 0 to 100. The operation information acquisition unit 305 then transmits that speed value to the parameter creation unit 306. The mapping method from operation information to speed value can be a simple normalization. That is, the speed value can be calculated by multiplying the value indicated by the operation information by the width of the PTZ value (e.g., 100 if the value range is 0 to 100, 80 if the region is 20 to 100). Alternatively, for example, the value indicated by the operation information may be divided into ranges of 0.0 to 0.5, 0.5 to 0.8, and 0.8 to 1.0, and then normalized to 0 to 20, 20 to 70, and 70 to 100, respectively. In other words, when the value "x" indicated by the operation information is between 0.0 and 0.5, the speed information can be represented by 20(x / 0.5). Also, when x is between 0.5 and 0.8, the speed information can be represented by 50{(x-0.5) / 0.3}+20, and when x is between 0.8 and 1.0, the speed information can be represented by 30(x-0.8) / 0.2+70. This allows for high-speed tracking by finely controlling the PTZ value in the low-speed range and quickly changing the PTZ value in the high-speed range. Note that these are just examples, and for example, operation information may be mapped to speed values ​​based on a value range (e.g., 0 to 50) that corresponds to a part of the PTZ value range. In other words, the operation information acquisition unit 305 may map operation information in the range of 0.0 to 1.0 to speed values ​​of 0 to 50. In one example, the mapping to speed values ​​may be performed using the distance between the current PTZ value and the PTZ value of the next first target as the maximum value (i.e., if the distance is 10, only speed values ​​between 0 and 10 are used).

[0040] Subsequently, in S711, the parameter creation unit 306 creates parameters for controlling the imaging device 101 based on the speed value acquired in S710 and the first target. The parameter creation unit 306 uses, for example, the following equations (2) and (3) to determine the pan value movement speed Speed pan And, the tilt movement speed tilt Calculate.

number

[0041] Furthermore, the parameter creation unit 306 can calculate the zoom speed using the following equation (4).

number

[0042] In this example, parameters are calculated based on the current camera information and the first information, but the parameters may also be calculated using the second information instead of the first information. The parameter creation unit 306 creates control parameters for controlling the imaging device 101 based on the calculated PTZ speed and the second target, and supplies the created control parameters to the imaging device control unit 307.

[0043] The parameter creation unit 306 adds, for example, the speed value calculated by equations (2) to (4) above to the PTZ value based on preset information. For example, let PTZ(t) be a vector representing the PTZ value (pan value, tilt value, zoom value) based on preset information at time t, and let the actual PTZ value at that time be PTZ(t) + ΔPTZ(t). If the PTZ value based on preset information at time t + Δt is PTZ(t + Δt), then if there is no manipulation of the speed value, the PTZ value of the imaging device 101 at that time will be PTZ(t + Δt) + ΔPTZ(t). On the other hand, if there is manipulation of the speed value, the PTZ value of the imaging device 101 at that time will be PTZ(t + Δt) + ΔPTZ(t) + Speed(t + Δt). Here, Speed(t + Δt) is a vector whose elements are the speed values ​​of the pan value, tilt value, and zoom value calculated by equations (2) to (4). The cumulative value of the speed up to time t is expressed as ΔPTZ(t), and ΔPTZ(t+Δt) = ΔPTZ(t) + Speed(t+Δt). Note that Δt may be, for example, the time length corresponding to one frame, or it may be an arbitrarily determined predetermined time length, such as the time length corresponding to a predetermined number of frames. In one example, Δt may be set to the time length at which the preset information is updated if the speed of PTZ value change does not change. Note that if Δt is sufficiently long, such as several seconds, the parameter creation unit 306 may gradually change the PTZ value so that at time t+Δt the PTZ value is calculated as described above. For example, if the number of frames corresponding to Δt is m, the PTZ value corresponding to each frame may be calculated so that the PTZ value changes by {PTZ(t+Δt)-PTZ(t)+Speed(t+Δt)} / m for each frame.

[0044] Subsequently, in S712, the imaging device control unit 307 transmits the control parameters created in S711 to the imaging device 101.

[0045] The user may, for example, slow down the speed at which the PTZ value moves by pressing button 621 on controller 103. In this case, the speed of the delay is defined in the same way as when button 601 is operated. For example, Speed(t+Δt) will be the value obtained by reversing the sign (positive or negative) of the value when button 601 is operated. If Speed(t+Δt) becomes PTZ(t)-PTZ(t+Δt), the PTZ value will not change between time t and time t+Δt. For this reason, Speed(t+Δt) may be calculated with the value of PTZ(t)-PTZ(t+Δt) as the limit value so as not to reverse the order of the preset positions. Button 621 does not have to be used. For example, the speed of changing the PT value may be increased by pressing button 601 while pressing button 614, and the speed of changing the PT value may be decreased by pressing button 601 while pressing button 613. Furthermore, the relationship between the button operation described above and the control performed in the information processing device 102 is merely an example, and other relationships may be used. The zoom value may also be fixed, in which case the zoom value does not need to be considered in each of the processes described above.

[0046] As described above, it becomes possible to take photos smoothly and appropriately in response to user operations, without deviating from the pre-set paths provided as preset information, while tracking the subject.

[0047] This disclosure can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0048] (Summary of the embodiments) At least some of the embodiments described above can be summarized as follows: (Item 1) An acquisition means for acquiring information indicating multiple setting values ​​that set the field of view, which correspond to multiple positions included in a predetermined path when taking images while moving the field of view of the imaging device along a predetermined path, A setting means for setting a target value for setting the field of view, which is different from the aforementioned multiple setting values, based on two setting values ​​included in the aforementioned multiple setting values, A means for changing the settings of the imaging device toward the target value, An information processing device characterized by having the following features. (Item 2) The system has an acquisition means for acquiring information indicating the field of view of the imaging device, The information processing apparatus according to item 1, characterized in that the setting means updates the target value based on a third setting value different from the first and second setting values ​​included in the plurality of setting values, based on the fact that the difference between the setting of the imaging device and the second setting value of the two setting values ​​becomes less than or equal to a predetermined value while the field of view of the imaging device is being changed from the first setting value of the two setting values ​​toward the target value. (Item 3) The information processing device according to item 2, wherein the setting means sets the target value based on the first setting value and the second setting value such that the position corresponding to the target value is set behind the position in the predetermined path corresponding to the second setting value, when viewed from the position in the predetermined path corresponding to the first setting value. (Item 4) The information processing device according to any one of items 1 to 3, characterized in that the settings of the imaging device, the plurality of setting values, and the target value each include at least the pan value and tilt value of the imaging device. (Item 5) The information processing device according to item 4, characterized in that the settings of the imaging device, the plurality of setting values, and the target value each include the zoom value of the imaging device. (Item 6) The system further includes receiving means for receiving operation information that indicates at least user operations related to the speed at which the settings of the imaging device are changed toward the target value, The modification means controls the speed at which the settings of the imaging device are changed based on the operation information. An information processing device according to any one of items 1 to 5, characterized by the above. (Item 7) The receiving means further receives operation information indicating at least user operations related to the zoom value in the settings of the imaging device, The modification means modifies the zoom value in the settings of the imaging device based on the operation information. The information processing device described in item 6, characterized by the features described herein. (Item 8) The receiving means further receives predetermined operation information regarding whether or not to perform a process to change the settings of the imaging device so that the field of view moves along the predetermined path. The modification means modifies the settings of the imaging device while the process is being performed, and does not modify the settings of the imaging device while the process is not being performed. An information processing device according to item 6 or 7, characterized by the features described therein. (Item 9) A control method performed by an information processing device, To acquire information indicating multiple setting values ​​that set the field of view, which correspond to multiple positions included in a predetermined path when taking images while moving the field of view of the imaging device along a predetermined path, Based on two of the aforementioned multiple setting values, a target value for setting the field of view, which is different from the aforementioned multiple setting values, is set. The setting of the imaging device is changed to the target value, A control method characterized by including (Item 10) A program to cause a computer to function as one of the means of an information processing device described in any one of items 1 through 8. This disclosure is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of this disclosure. [Explanation of Symbols]

[0049] 101: Imaging device, 102: Information processing device, 302: First target group creation unit, 303: Second target group creation unit, 307: Imaging device control unit

Claims

1. An acquisition means for acquiring information indicating multiple setting values ​​that set the field of view, which correspond to multiple positions included in a predetermined path when taking images while moving the field of view of the imaging device along a predetermined path, A setting means for setting a target value for setting the field of view, which is different from the aforementioned multiple setting values, based on two setting values ​​included in the aforementioned multiple setting values, A means for changing the settings of the imaging device toward the target value, An information processing device characterized by having the following features.

2. The system has an acquisition means for acquiring information indicating the field of view of the imaging device, The information processing apparatus according to claim 1, characterized in that the setting means updates the target value based on a third setting value different from the first and second setting values ​​included in the plurality of setting values, based on the fact that the difference between the setting of the imaging device and the second setting value of the two setting values ​​becomes less than or equal to a predetermined value while the field of view of the imaging device is being changed from the first setting value of the two setting values ​​toward the target value.

3. The information processing apparatus according to claim 2, characterized in that the setting means sets the target value based on the first setting value and the second setting value such that the position corresponding to the target value is set behind the position in the predetermined path corresponding to the second setting value, when viewed from the position in the predetermined path corresponding to the first setting value.

4. The information processing apparatus according to claim 1, characterized in that the settings of the imaging device, the plurality of setting values, and the target value each include at least the pan value and tilt value of the imaging device.

5. The information processing apparatus according to claim 4, characterized in that the settings of the imaging device, the plurality of setting values, and the target value each include the zoom value of the imaging device.

6. The system further includes receiving means for receiving operation information that indicates at least user operations related to the speed at which the settings of the imaging device are changed toward the target value, The modification means controls the speed at which the settings of the imaging device are changed based on the operation information. The information processing apparatus according to feature 1.

7. The receiving means further receives operation information indicating at least user operations related to the zoom value in the settings of the imaging device, The modification means modifies the zoom value in the settings of the imaging device based on the operation information. The information processing apparatus according to feature 6.

8. The receiving means further receives predetermined operation information regarding whether or not to perform a process to change the settings of the imaging device so that the field of view moves along the predetermined path. The modification means modifies the settings of the imaging device while the process is being performed, and does not modify the settings of the imaging device while the process is not being performed. The information processing apparatus according to feature 6.

9. A control method performed by an information processing device, To acquire information indicating multiple setting values ​​that set the field of view, which correspond to multiple positions included in a predetermined path when taking images while moving the field of view of the imaging device along a predetermined path, Based on two of the aforementioned multiple setting values, a target value for setting the field of view, which is different from the aforementioned multiple setting values, is set. The setting of the imaging device is changed to the target value, A control method characterized by including

10. A program for causing a computer to function as one of the means of the information processing device described in any one of claims 1 to 8.

Images