Information processing device, control method, and program

Abstract

Trace information can be edited while viewing the captured image. [Solution] An acquisition means for acquiring captured images from an imaging device capable of panning and tilting, An information processing apparatus comprising: a first information display area for displaying information on the transition of the pan-tilt position of the imaging apparatus; a second information display area for displaying at least one of the time transitions of the pan position and the time transitions of the tilt position; and a third information display area for displaying an image acquired by the acquisition means; and a control means for controlling the pan-tilt position of the imaging apparatus, wherein the control means controls the apparatus to drive to a pan-tilt position specified by the user from among the first information display area or the second information display area.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a control method, and a program.

Background Art

[0002] In recent years, network cameras and control devices for video production that can control imaging devices remotely have become widespread. As remote control of an imaging device, there are a pan-tilt-zoom operation and a preset operation in which a shooting position and settings are registered in advance and called up. Further, there is a trace operation in which the execution content and execution timing of a series of camera operations during a predetermined period are recorded as trace information, and control of the imaging device is executed by referring to the trace information.

[0003] When recording trace information, if the desired trace information cannot be recorded, there is a problem that the operation has to be redone from the beginning, including the parts that were successful.

[0004] Therefore, Patent Document 1 discloses a technique for correcting trace information by correcting a graph showing the locus of an imaging device based on the trace information.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, in the method described in Patent Document 1, when correcting trace information, it is impossible to confirm what kind of captured image will be obtained at the corrected pan-tilt-zoom position.

[0007] Therefore, the present invention aims to provide a technology that allows for editing trace information while viewing captured images. [Means for solving the problem]

[0008] The present invention relates to an information processing apparatus comprising: acquisition means for acquiring an image captured from an imaging device capable of panning and tilting; a first information display area for displaying transition information of the pan-tilt position of the imaging device; a second information display area for displaying the transition speed of at least one pan position and the transition speed of the tilt position; a display control means for controlling the display of a third information display area for displaying an image captured by the acquisition means; an editing means for editing the information displayed in the first and second information display areas; and a control means for controlling the pan-tilt position of the imaging device, wherein the control means controls the device to drive to a pan-tilt position specified by the user from among the first or second information display area. [Effects of the Invention]

[0009] According to the present invention, trace information can be edited while checking the captured image. [Brief explanation of the drawing]

[0010] [Figure 1] Diagram showing the system configuration according to the first embodiment. [Figure 2] This figure shows the hardware configuration of the imaging device 100 and the information processing device 200 according to the first embodiment. [Figure 3] A diagram showing the functional blocks of the information processing device 200 according to the first embodiment. [Figure 4] Figure showing an example of a GUI according to the first embodiment. [Figure 5] A diagram showing an example of trace information according to the first embodiment. [Figure 6] Flowchart showing the control flow according to the first embodiment [Figure 7] Figure showing an example of a GUI according to the first embodiment. [Figure 8] Figure showing an example of the GUI during trace information editing according to the first embodiment [Figure 9] Figure showing an example of the graph of the locus line according to the first embodiment [Figure 10] Flowchart showing the control flow according to the second embodiment [Figure 11] Figure showing an example of the trace information according to the second embodiment [Figure 12] Figure showing an example of the GUI according to the third embodiment [Figure 13] Flowchart showing the control flow according to the third embodiment [Figure 14] Flowchart showing the control flow according to the third embodiment [Figure 15] Flowchart showing the control flow according to the fourth embodiment [Figure 16] Figure showing an example of the GUI according to the fourth embodiment

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant explanations are omitted.

[0012] (First Embodiment) <System Configuration> FIG. 1 is a diagram showing a configuration example of an imaging system including a control device 200 according to the present embodiment. The system according to the first embodiment has an imaging device 100 and an information processing device 200. The imaging device 100 and the information processing device 200 are interconnected via a network 300. The method of connection between the respective devices is not limited to a specific method. For example, they may be connected by a wired cable.

[0013] The imaging device 100 transmits, as signals via the network 300, image data based on the imaging results and various information of the imaging device 100 to the information processing device 200. Here, for the sake of explanation, it is assumed that the information processing device 200 is connected to the imaging device 100, but there is no limit to the number of imaging devices to be connected, and there may be one or more imaging devices.

[0014] The signal input to the information processing device 200 is not limited to the case where the imaging device 100 is directly connected, and it is also possible to input a signal via other processing devices such as a switch and an effector.

[0015] In addition, the information processing device 200 can also perform operations such as calling and deleting preset functions and detailed setting operations of the automatic loop function.

[0016] In addition, the information processing device 200 is described as being integrated with hardware having an operation unit that enables operations on a camera or an output image, such as a camera controller, but it may be separate and independent.

[0017] When the information processing device 200 is independent of the operation unit, a control device such as a camera controller is used as the operation unit to set the position of the PTZ (pan-tilt-zoom) drive unit of the imaging device and other settings such as image quality, and it is possible to register them as presets. In this case, the controller, which is the operation unit, is connected to the imaging device 100 via the network 300 or connected using a serial connection or the like.

[0018] The network 300 is a network that connects the imaging device 100 and the information processing device 200. The network 300 is realized from a plurality of routers, switches, cables, etc. that conform to a communication standard such as ETHERNET (registered trademark). Note that the network 300 may be realized by the Internet, a wired LAN (LOCAL AREA NETWORK), a wireless LAN, a WAN (WIDE AREA NETWORK), or the like.

[0019] <Hardware configuration of the imaging device> Next, the imaging device 100 according to this embodiment will be described using Figure 2. Figure 2 is a diagram showing an example of the hardware configuration of the imaging device 100. The imaging device 100 includes an imaging unit 101, a lens drive unit 102, a pan drive unit 103, a tilt drive unit 104, an image processing unit 105, a zoom control unit 106, and a pan / tilt control unit 107. Furthermore, it includes a system control unit 108, a storage unit 109, and a communication unit 110, and each component is connected to communicate with each other via an internal bus (not shown).

[0020] The programs for realizing the functions according to this embodiment, and the data used when such programs are executed, are stored, for example, in the ROM of the storage unit 109. Furthermore, these programs and data are loaded, for example, into the RAM of the storage unit 109 under the control of the system control unit 108 and executed by the system control unit 108.

[0021] The imaging unit 101 captures an image of the subject and converts it into an electrical signal. Specifically, the imaging unit 101 includes an imaging optical system and an image sensor. The imaging optical system consists of lenses that focus light from the subject onto the imaging surface of the image sensor, and is composed of, for example, a zoom lens, a focus lens, and an image stabilization lens. The image sensor captures the image of the subject incident through the imaging optical system and converts the image into an electrical signal. The image sensor is implemented by, for example, a CCD sensor or a CMOS sensor. Part of the imaging optical system also functions as a zoom drive unit 102, which will be described later. Furthermore, the exposure can be adjusted by changing the shutter speed and gain of the image sensor.

[0022] The lens drive unit 102 consists of a drive system for the focus lens and the zoom lens, and its operation is controlled by the zoom control unit 106. Furthermore, the zoom lens and focus lens are driven by control commands from the system control unit 108 (described later) obtained via the zoom control unit 106. By driving the zoom lens and focus lens, the zoom value and focus value can be changed. In addition, the lens drive unit 102 outputs zoom value and focus value information to the zoom control unit 106. The output zoom value and focus value may be expressed as values ​​such as zoom magnification and focal length, or as the focal position.

[0023] The pan drive unit 103 consists of a mechanical drive system and a motor that perform the pan operation, and its operation is controlled by the pan-tilt control unit 107. The pan drive unit 103 consists of an actuator such as a stepping motor and an encoder that detects the pan position. The detected pan position (pan value) can be transmitted to the pan-tilt control unit 107.

[0024] The tilt drive unit 104 consists of a mechanical drive system and a motor that perform the tilt operation, and its operation is controlled by the pan-tilt control unit 107. The tilt drive unit 104 consists of an actuator such as a stepping motor and an encoder that detects the tilt position. The detected tilt position (tilt value) can be transmitted to the pan-tilt control unit 107.

[0025] The image processing unit 105 performs predetermined image processing and compression encoding processing on the signal captured and photoelectrically converted by the imaging unit 101 to generate image data. Note that the generated image data is not limited to one; multiple image data with different resolutions and video qualities may be generated simultaneously. The generated image data is then output to the system control unit 108.

[0026] The zoom control unit 106 controls the lens drive unit 102 based on control commands obtained from the system control unit 108.

[0027] The pan-tilt control unit 107 controls the pan drive unit 103 and the tilt drive unit 104 based on control commands received from the system control unit 108.

[0028] The system control unit 108 is an example of a processor that comprehensively controls the entire imaging device 100, represented by a CPU (Central Processing Unit). In this embodiment, the system control unit 108 analyzes control commands acquired via the communication unit 110 and performs processing according to the commands. Specifically, it outputs control commands for image quality adjustment to the image processing unit 105, control commands for zoom and focus control to the zoom control unit 106, and control commands for pan and tilt operation to the pan and tilt control unit 107. It also controls the zoom control unit 106 and the pan and tilt control unit 107 to store the outputted control commands in the storage unit 109. At this time, it is controlled to store the outputted control commands in the storage unit 109 in association with time information, but it is not limited to this. For example, only the control commands may be stored.

[0029] Furthermore, the system control unit 108 acquires the image data generated by the image processing unit 105 and transmits it to the communication unit 110. The system control unit 108 also records the control commands acquired during the creation and editing of trace information (transition information), which will be described later, in the storage unit 109, associating them with time information (target arrival time). In this embodiment, the system control unit 108 stores the trace information, but this is not limited to this. For example, the system control unit 204, which will be described later, may store the control commands input to the input unit 202 during the creation and editing of trace information in the storage unit 203, associating them with time information. Here, the time information included in the trace information is information that shows the time transition of the position of each trajectory point. The target arrival time is the elapsed time from the playback of the trace until the specified pan-tilt position is reached.

[0030] The memory unit 109 stores image data, various setting commands, and the like. The memory unit 109 stores various computer programs and various data as information necessary for the system control unit 108 to perform processing. In other words, the memory unit 109 is the main memory and is used as a temporary storage area for the work area and various programs. The memory unit 109 may also be used as a storage area for temporarily holding various data such as image data corresponding to the imaging results.

[0031] Furthermore, the memory unit 109 is a non-volatile memory unit, such as a flash memory, HDD, SSD (Solid State Drive), or SD card. The memory unit 109 is also used as a persistent memory area to store the OS, various programs, and various data, including programs that the system control unit 108 uses to control the imaging device 100. In addition, it is also used as a short-term memory area to store various setting parameters.

[0032] The communication unit 110 transmits image data sent from the system control unit 108 to the information processing device 200. It also receives various setting commands and control commands sent from the information processing device 200 and outputs them to the system control unit 108. Furthermore, it transmits the responses of the imaging device 100 to the various commands obtained from the information processing device 200 to the information processing device 200. In other words, it is an interface for connecting the aforementioned network 300 and the aforementioned components, and is responsible for communication with external devices such as the image processing device 200 via a communication medium such as Ethernet (registered trademark). Here, the control of the imaging device 100 may be performed via the communication unit 110, or via another interface such as a serial communication interface (not shown).

[0033] In this embodiment, the imaging device 100 is illustrated by an example where one system control unit 108 uses one storage unit 109 to execute each process shown in the flowchart described later. However, this is not necessarily required. For example, multiple system control units and storage units may cooperate to execute each process shown in the flowchart described later. Also, at least a part of each process shown in the flowchart may be executed by dedicated hardware. Examples of dedicated hardware include ASICs (Application Specific Integrated Circuit Units) and FPGAs (Field-Programmable Gate Arrays). Furthermore, the processor is not limited to a CPU. The processor may be, for example, a GPU (Graphics Processing Unit).

[0034] <Hardware configuration of the information processing device> Next, an example of the hardware configuration of the information processing device 200 will be explained in detail using Figure 2.

[0035] The image processing device 200 includes a display unit 201, an input unit 202, a storage unit 203, a system control unit 204, and a communication unit 205, and each component is connected to communicate with the others via an internal bus (not shown). Programs for realizing the functions according to this embodiment and data used when such programs are executed are stored, for example, in the ROM of the storage unit 203. Furthermore, these programs and data are loaded, for example, into the RAM of the storage unit 203 under the control of the system control unit 204 and executed by the system control unit 204.

[0036] The information processing device 200 can be a general-purpose computer such as a Personal Computer (hereinafter referred to as a PC), or a mobile terminal such as a tablet. In other words, the information processing device 200 will be described as a general client terminal having a display unit such as a PC or tablet and an operating unit, but it is not limited to these. The terminal having a display unit such as a display and the operating unit of the information processing device 200 may be separate units.

[0037] The display unit 201 uses a display device such as an LCD projector or LCD monitor to display images acquired from the imaging device 100 and to display a Graphic User Interface (hereinafter referred to as GUI) for controlling the imaging device 100. The display unit 201 also displays a GUI for creating, editing, and playing back trace information, which will be described later.

[0038] The input unit 202 is a User Interface, such as a keyboard, mouse, or touch panel, enabling the exchange of information between the user and the information processing device 200. Examples of the input unit 202 include an output device such as a display and an input device such as a keyboard, mouse, or touchpad. The input unit 202 receives input information from the user and transmits it to the system control unit 204. In this embodiment, the user operates the GUI displayed on the display unit 201 via the input unit 202. The user can also input control information from the input unit 202 to change the imaging range of the imaging device 100. The control method is not limited to a specific method; for example, an input device such as a joystick may be connected to the input unit 202. When the input unit 202 acquires control information to change the imaging range of the imaging device 100, it outputs it to the system control unit 204.

[0039] The memory unit 203 records camera information and other data to internal and external storage. The memory unit 203 stores various computer programs and various data as information necessary for the system control unit 204 to perform processing. In other words, the memory unit 203 is the main memory and is used as a temporary storage area for the work area and various programs. The memory unit 203 may also be used as a storage area for temporarily holding various data such as image data acquired from the imaging device.

[0040] Furthermore, the memory unit 203 is a non-volatile memory unit, such as flash memory, HDD, SSD (Solid State Drive), or SD card. The memory unit 203 is also used as a persistent memory area to store the OS, various programs, and various data, as well as programs that the system control unit 204 uses to control the information processing device 200. In addition, it is also used as a short-term memory area to store various setting parameters.

[0041] The system control unit 204 is an example of a processor that provides overall control of the information processing device 200, which is represented by a CPU (Central Processing Unit). In this embodiment, the system control unit 204 controls the information processing device 200 in response to user GUI operations acquired via the input unit 202. Furthermore, if the user's GUI operation relates to the imaging device 100, it generates various setting commands and control commands and transmits them to the imaging device 100 via the communication unit 205. It also receives the response from the imaging device 100 to the transmitted setting commands and control commands via the communication unit 205. In addition, it controls the information processing device 200 to display the image data received from the imaging device 100 via the communication unit 205 on the display unit 201.

[0042] The communication unit 205 transmits various setting commands and control commands sent from the system control unit 204 to the imaging device 100. It also transmits image data transmitted from the imaging device 100 and the imaging device 100's response to commands transmitted from the information processing device 200 to the system control unit 204. In other words, it is an interface for connecting the aforementioned network 300 and the aforementioned components, and is responsible for communication with external devices such as the imaging device 100 via a communication medium such as Ethernet®.

[0043] In this embodiment, the information processing device 200 is illustrated by an example where one system control unit 204 uses one storage unit 203 to execute each process shown in the flowchart described later. However, this is not necessarily required. For example, multiple system control units and storage units may cooperate to execute each process shown in the flowchart described later. Also, at least a part of each process shown in the flowchart may be executed by dedicated hardware. Examples of dedicated hardware include ASICs (Application Specific Integrated Circuit Units) and FPGAs (Field-Programmable Gate Arrays). Furthermore, the processor is not limited to a CPU. The processor may be, for example, a GPU (Graphics Processing Unit).

[0044] In this way, the information processing device 200 can control the imaging device 100 by outputting control commands to the imaging device 100 via the network 300.

[0045] <Functional block diagram of information processing device> Next, the functions of the system control unit 204 of the information processing device 200 will be explained with reference to Figure 3. Figure 3 is a diagram showing an example of the configuration of functions included in the system control unit 204 of the information processing device 200. As shown in Figure 3, the system control unit 204 consists of a camera information acquisition unit 220, an image acquisition unit 221, a PTZ control unit 222, a user request acquisition unit 223, a trace editing unit 224, a trace playback unit 225, a trajectory calculation unit 226, and a trace information drawing unit 227. Each function shown in Figure 3 is realized by the system control unit 204 reading programs stored in the memory unit into the memory unit 203 and executing them. In other words, the system control unit 204 uses these programs to perform image processing and data calculations. The system control unit 204 also executes programs stored in the memory unit 203, manages tasks such as reading, processing, and outputting image data, and realizes the operation of the entire system.

[0046] The camera information acquisition unit 220 transmits an information acquisition command to the imaging device 100 via the communication unit 205. It also acquires information from the imaging device 100 and outputs it to the storage unit 203. The information that the camera information acquisition unit 220 can acquire is the zoom value, pan value, and tilt value (current value) of the lens drive unit 102, pan drive unit 103, and tilt drive unit 104. Furthermore, the camera information acquisition unit 220 can acquire information such as the movable range of the lens drive unit 102, pan drive unit 103, and tilt drive unit 104. In addition, if the information processing device 200 is playing back trace information, it outputs the zoom value, pan value, and tilt value (current value) of the lens drive unit 102, pan drive unit 103, and tilt drive unit 104 to the trace information drawing unit 227.

[0047] The image acquisition unit 221 transmits an image acquisition command to the imaging device 100 via the communication unit 205, acquires a captured image from the imaging device 100, and outputs it to the display unit 201.

[0048] The PTZ control unit 222 generates PTZ control commands via the communication unit 205 based on control information input from the trace editing unit 224 or the trace playback unit 225, or control information input via the input unit 202, and transmits them to the imaging device 100. Here, the PTZ control command is a control command for controlling the lens drive unit 102, pan drive unit 103, and tilt drive unit 104 of the imaging device 100. The imaging device 100 controls the lens drive unit 102, pan drive unit 103, and tilt drive unit 104 of the imaging device 100 based on the control commands transmitted from the PTZ control unit 222. Here, when playing back trace information, start and stop commands for trace control are sent to the imaging device 100, and the control of the lens drive unit 102, pan drive unit 103, and tilt drive unit 104 during trace operation may be performed internally by the imaging device 100. In that case, it is assumed that the trace information is stored in the storage unit 109 of the imaging device 100.

[0049] The user request acquisition unit 223 receives user requests output from the input unit 202. User requests include "trace recording request," "trace editing request," and "trace playback request." Furthermore, "trace recording request" includes "start recording" and "stop recording." "Trace editing request" includes one of the following request information: "add point," "edit point," or "delete point." "Trace playback request" includes one of the following request information: "start playback" or "stop playback." If the user request is a "trace recording request" or a "trace editing request," it is output to the trace editing unit 224. If the user request is a "trace playback request," the request information is output to the trace playback unit 225.

[0050] The trace editing unit 224 records trace information if the request information input from the user request acquisition unit 223 is a trace recording request and includes a recording start. The trace editing unit 224 stops recording trace information if the request information input from the user request acquisition unit 223 is a trace recording request and includes a recording stop. Similarly, if the request information input from the user request acquisition unit 223 is a trace editing request, the trace editing unit 224 reads the trace information stored in the storage unit 203 and edits the trace information. If the request information is "add point", the trace editing unit 224 adds a new trajectory point to the trace information based on the parameter corresponding to the position where the graph operation was performed.

[0051] In this embodiment, a trajectory point is a point in the trace information that has a pan-tilt position (target position) that the user specifies and wants to be reached. Furthermore, it is a point that has information (identification information) about the elapsed time from the start of the trace (the time from the playback of the trace until the pan-tilt position is reached). In addition, a trajectory point is a point that has parameters for calculating the trajectory line connecting the trajectory point and the next trajectory point, and the control points that constitute the trajectory line. That is, a control point is a point that constitutes the trajectory line connecting the trajectory point and the next trajectory point. A control point is a parameter that includes information about the pan-tilt position and the elapsed time. In other words, a trajectory point is a parameter set by the user, and a control point is a parameter calculated from the trajectory point set by the user. The control points are determined by performing piecewise cubic Hermitian interpolation using two or three control points. Note that the interpolation method used when determining the control points can be any interpolation method that can obtain the information necessary to smoothly connect preceding and succeeding trajectory points, such as cubic spline interpolation or Akima interpolation. The control points are determined based on the trajectory points entered in this way. The method for calculating the control points and trajectory lines will be described later.

[0052] In this embodiment, the trajectory point is described as a point with a pan-tilt position, but it is not limited to this. For example, it may have parameters related to the zoom position. In that case, the zoom position information held by the trajectory point is the zoom position that the user specifies and wants to be reached.

[0053] If the request information input from the user request acquisition unit 223 is "point editing", the trace editing unit 224 changes the PT(Z) position or elapsed time related to the trajectory point to be edited, or changes the parameters for calculating the trajectory line to be edited. If the request information input from the user request acquisition unit 223 is "point deletion", the trace editing unit 224 deletes the trajectory point being edited from the trace information.

[0054] The trace playback unit 225 reads the trace information stored in the storage unit 203 from the request information input from the user request acquisition unit 223 and executes the trace playback process. If the request information is "start playback", it outputs control information to the PTZ control unit 222 and time information to the trace information drawing unit 227. If the request information is "stop playback", it stops the information output to the PTZ control unit 222 and the trace information drawing unit 227 and cancels playback.

[0055] The trajectory calculation unit 226 calculates trajectory lines connecting the points of each trajectory from the trace information described later. In this embodiment, the time-pan trajectory line, which represents the relationship between time and pan, and the time-tilt trajectory line, which represents the relationship between time and tilt, are each calculated using cubic Bézier curves. In this embodiment, cubic Bézier curves are used to calculate the trajectory lines, but other trajectories that can represent the transitions of time, pan, and tilt may be used, such as cubic spline curves or Lagrangian interpolation polynomials.

[0056] The trace information drawing unit 227 creates the content to be displayed on the display unit 201 based on the time information and control information input from the camera information acquisition unit 220 and the trace playback unit 225. The content to be displayed will be described later.

[0057] <How to create and edit trace information> Next, with reference to Figure 4, the method for creating and editing trace information in the first embodiment will be described. The GUI400 in Figure 4 is a settings screen for creating, editing, and playing back trace information displayed on the display unit 201 of the information processing device 200.

[0058] The GUI 400 in Figure 4 includes a trace ID 401, which is information for identifying trace information, and a start recording button 402 for instructing the start of recording (creation) of trace information. Furthermore, it includes a stop recording button 403 for instructing the cessation of recording trace information, and a play button 404 for instructing the playback and cessation of trace information. In addition, when editing trace information, it includes a point add button 405, a point edit button 406, and a point delete button 407 for switching the editing state of the trace information. Furthermore, it includes a pan-tilt graph 410 (first information display area), a two-dimensional graph showing the relationship between pan value and tilt value. It also includes a time-pan graph 420 (second information display area), a two-dimensional graph showing the relationship between pan value and time, and a time-tilt graph 430 (second information display area), a two-dimensional graph showing the relationship between tilt value and time.

[0059] Here, the time shown on the horizontal axis of the time-pan graph 420 and the time-tilt graph 430 represents the elapsed time from the point in time when the playback of the trace information began. The GUI 400 in Figure 4 displays graphs based on the created trace information. In this embodiment, the pan-tilt graph 410, the time-pan graph 420, and the time-tilt-time graph 430 are displayed, but this is not limited to them. For example, a graph related to the zoom value (time-zoom graph) may be displayed. Alternatively, a three-dimensional graph showing the pan value, tilt value, and zoom value may be displayed.

[0060] In this way, users can create and edit trace information by performing operations on the displayed graph.

[0061] Trace ID 401 is an ID (hereinafter referred to as Trace ID) that uniquely identifies a series of trace information created. In the example in Figure 4, when the user clicks on Trace ID 401, a list of Trace IDs corresponding to each of the registered trace information entries is displayed. Additionally, Trace ID 401 allows for the selection of "Add Trace ID" (not shown), and selecting this will generate a new Trace ID, which will then be displayed in the selected state. Note that Trace ID 401 may also have a fixed list of, for example, 10 Trace IDs.

[0062] The recording start button 402 is a button for creating trace information, and the creation of trace information begins when the user presses the recording start button 402.

[0063] The recording stop button 403 is used to stop the recording of trace information that is being created. When the user presses the recording start button, the creation of trace information is stopped, and the storage unit 203 records the trace information from the start to the stop of trace information creation, associating it with the trace ID selected by trace ID 401. In this embodiment, the recording start button 402 and the recording stop button 403 are used when creating trace information by controlling the imaging device described later, but are not limited to this. They may also be used when creating trace information by graph operations described later.

[0064] The play button 404 is used to play back the created trace information. When the user presses the play button 404, the system control unit 204 reads the trace information associated with the selected trace ID 401 from the storage unit 203 and performs the following process. Specifically, the system control unit 204 controls the imaging device 100 to trace the parameter changes by referring to the trace information. This process of controlling the imaging device 100 to trace the parameter changes caused by user operations by referring to the trace information is called trace playback. Furthermore, if the play button 404 is pressed again while it is already pressed, trace playback to the imaging device 100 is stopped.

[0065] The Add Point button 405 is used to add a trajectory point. When the user presses the Add Point button 405, a "Trace Edit Request" including "Add Point" is output to the User Request Acquisition Unit 223 via the Input Unit 202. If the user operates one of the graphs described later while the Add Point button 405 is pressed, a trajectory point is added to the corresponding position. In this way, the Trace Edit Unit 224 can add a new trajectory point to the trace information of the trace ID selected by Trace ID 401 based on the parameters corresponding to the position where the graph operation was performed.

[0066] The point editing button 406 is a button for editing trace information. When the user presses the point editing button 406, a "trace editing request" including "point editing" is output to the user request acquisition unit 223 via the input unit 202. By pressing the point editing button 406 and manipulating one of the graphs described later, the user can drag the trajectory line, trajectory points, or control points to change the target position, elapsed time, or control point position of the corresponding point.

[0067] The point erase button 407 is used to erase control points. When the user presses the point erase button 407, a "trace editing request" including "point erase" is output to the user request acquisition unit 223 via the input unit 202. If a point drawn on one of the graphs described later is selected while the point erase button 407 is pressed, the trace information of the corresponding point can be erased.

[0068] In this way, the point add button 405, the point edit button 406, and the point delete button 407 switch between a state where "point addition," "point editing," and "point deletion" are possible and a state where they are not, respectively, when pressed. Each button has two states: pressed and unpressed.

[0069] The pan-tilt graph 410 is a two-dimensional graph representing the relationship between pan and tilt in the trace information shown in Figure 4. When the add point button 405 is pressed, the user can add a trajectory point at any location on the pan-tilt graph 410 by clicking, for example, a mouse button. When the edit point button 406 is pressed, the user can manipulate any of the trajectory points or control points on the pan-tilt graph 410. Depending on the operation, the target position of the pan or tilt of the trajectory point, or the position of the control point can be changed. For example, the user can select a trajectory point by clicking a mouse button and then manipulate the trajectory point or control point by dragging it.

[0070] In this case, to make it easier to identify which trajectory point is selected, the selected trajectory point may be highlighted by changing its color, size, shape, etc. When the point deletion button 407 is pressed, the user can delete any trajectory point on the pan-tilt graph 410 by selecting it, for example, by clicking a mouse button.

[0071] In this embodiment, for example, if the elapsed time of the first trajectory point added on the pan-tilt graph 410 is set to zero, then when the next trajectory point is added, the elapsed time of the added trajectory point information is set to the elapsed time of the previous trajectory point plus a predetermined time.

[0072] Here, when a user adds a trajectory point, the elapsed time is set by adding a predetermined time to the elapsed time of the previous point, but this is not the only option. For example, the elapsed time from the time the point addition button 405 was pressed could be set.

[0073] The time-pan graph 420 is a two-dimensional graph showing the relationship between time and pan values ​​in the trace information shown in Figure 4. When the add point button 405 is pressed, the user can add a trajectory point at any location on the time-pan graph 420 by clicking, for example, a mouse button. At this time, the tilt value of the trajectory point will be recorded as zero, but this is not limited to that. For example, the user may input the same tilt value as the trajectory point closest to the added trajectory point, or the user may input the tilt value when adding the trajectory point. When the edit point button 406 is pressed, the user can select any trajectory point on the time-pan graph 420 by clicking, for example, a mouse button. By dragging the selected trajectory point or control point, the target pan position of the trajectory point, the elapsed time, or the position of the control point can be changed. When the delete point button 407 is pressed, the user can delete any trajectory point on the time-pan graph 420 by selecting it by clicking, for example, a mouse button.

[0074] The time-tilt graph 430 is a two-dimensional graph showing the relationship between time and pan in the trace information shown in Figure 4. When the add point button 405 is pressed, a trajectory point can be added to any location on the time-tilt graph 430 by clicking, for example, a mouse button. At this time, the pan value will be recorded as zero for the trajectory point, but this is not limited to that. For example, the same pan value as the nearest trajectory point to the added trajectory point may be entered, or the user may enter the pan value when adding the trajectory point. When the edit point button 406 is pressed, the user can select any trajectory point on the time-tilt graph 430 by clicking, for example, a mouse button. By dragging the selected trajectory point or control point, the target position of the trajectory point's tilt, the elapsed time, or the position of the control point can be changed. When the delete point button 407 is pressed, the user can delete any trajectory point on the time-tilt graph 430 by selecting it by clicking, for example, a mouse button.

[0075] In this way, by manipulating the graph area and adding the first trajectory point, information such as the pan value, tilt value, and elapsed time corresponding to the imaging device 100 are recorded as trajectory point information based on the trajectory point. In this way, by manipulating and editing a single two-dimensional graph, it becomes possible to set two parameters that can be specified on the two-dimensional graph, and parameters that cannot be specified on the two-dimensional graph, as trajectory point information.

[0076] In this embodiment, when a trajectory point is created on the pan-tilt graph 410, the position of the created trajectory point and the pan and tilt values ​​of the corresponding location on the pan-tilt graph 410 are recorded as trajectory point information. Furthermore, a preset elapsed time is also recorded as trajectory point information.

[0077] Similarly, when a trajectory point is created on the time-pangraph 420, the position of the created trajectory point, the pan value of the corresponding location on the time-pangraph 420, and the elapsed time are recorded as trajectory point information. Furthermore, a pre-set tilt value is recorded as trajectory point information. When a trajectory point is created on the time-tiltgraph 430, the position of the created trajectory point, the tilt value of the corresponding location on the time-tiltgraph 430, and the elapsed time are recorded as trajectory point information. Furthermore, a pre-set tilt value is recorded as trajectory point information.

[0078] Here, trace information may be recorded by controlling the imaging device 100, or it may be created by the user adding trajectory points to a two-dimensional pan-tilt graph 410 that shows the relationship between time and pan and tilt values ​​in the trace.

[0079] As described above, users can record trace information associated with a single trace ID by adding, editing, and deleting trace points, but this is not limited to this. For example, a time limit may be set in advance. In this case, if the elapsed time from the time the first trace point was added reaches the set time, the storage unit 203 may stop recording traces and store the sequentially recorded trace information in the trace information table. Also, if a predetermined time has elapsed since the time a trace point was added, edited, or deleted, the system may stop recording traces and store the sequentially recorded trace information in the trace information table. The user may set and change this time, or it may be determined in the initial settings.

[0080] Note that the point addition / editing / deletion operations described above are not limited to graph manipulation; they can also be achieved, for example, by numerical input.

[0081] Furthermore, similar control can be applied when editing already created trace information. In this case, the user specifies the trace ID they wish to edit using trace ID 401, and the trace information stored in the storage unit 203 is read and drawn by the trace information drawing unit 227. The user can edit the trace information using the same operations as when creating the trace information. The specific control flow will be described later.

[0082] Next, we will explain how to create trace information by controlling the imaging device 100.

[0083] When the user presses the recording start button 402 and the creation of trace information begins, the user performs operations on the imaging device 100 to create the trace information. When the creation of trace information has started, the user can press the recording stop button 403 to stop the creation of trace information. Furthermore, the storage unit 203 records the trace information from the start to the stop of trace information creation, associating it with the trace ID selected by trace ID 401.

[0084] First, we will explain the operation of controlling the imaging device 100 to record trace information. Here, when the user inputs control information to control the imaging device 100 from the input unit 202, control commands for the imaging device are generated sequentially in response to the user operation, and these control commands are transmitted to the imaging device 100, thereby realizing control of the imaging device 100. For example, if a joystick is connected to the input unit 202, when the user performs an operation to tilt the joystick left or right, the PTZ control unit 222 generates a control command in response to the input user operation and transmits it to the imaging device 100.

[0085] Furthermore, if the user performs an operation to tilt the joystick in the vertical direction, the PTZ control unit 222 generates a control command to change the tilt value and transmits it to the imaging device 100. In recording trace information, the recording unit 203 records the transition of parameters of the imaging device 100, which are sequentially changed by user operation, and the corresponding time information (time transition of pan and tilt position) as trace information linked to the currently selected trace ID. In this embodiment, when recording trace information by controlling the user's imaging device 100, it is explained as recording the transition of parameters at a predetermined period. In that case, the pan value, tilt value, and zoom value at a predetermined period are recorded as corresponding trajectory points. That is, trace information is recorded with the pan value, tilt value, and zoom value at, for example, every 2 seconds after the recording start button 402 is pressed, as trajectory points.

[0086] The information processing device 200 records the changes in parameters of the sequentially added trajectory points and the corresponding time information (time changes in pan and tilt positions) as trace information linked to the currently selected trace ID until the recording stop button 403 is pressed.

[0087] As described above, trace information is created when the user presses the recording start button 402 and performs operations on the imaging device 100. In other words, the information processing device 200 records the changes in the parameters of the imaging device 100 as they are sequentially changed, along with the corresponding time information, as trace information linked to the currently selected trace ID until the recording stop button 403 is pressed.

[0088] In this embodiment, the system is described as having two methods for creating trace information: controlling the imaging device 100, and using the point addition button 405, point editing button 406, and point deletion button 407. However, it is not limited to these two methods. A GUI for selecting which method to use to record trace information may be displayed on the display unit 201, or only one of the methods may be provided.

[0089] (Replaying trace information) When the play button 404 is pressed by the user, the system control unit 204 reads the trace information associated with the selected trace ID 401 from the storage unit 203 and performs the following processing. When the play button 404 is pressed, a "start playback" trace playback request is sent to the user request acquisition unit 223, and it becomes a button for sending a trace playback stop request. In other words, if the play button 504 is pressed while trace information is being played back, a "stop playback" trace request is sent to the user request acquisition unit 223. Also, when the playback of trace information is finished, it transitions to the stopped state and becomes a play button to send a trace playback start request.

[0090] The trace information table will now be explained with reference to Figure 5. The trace information table 500 shown in Figure 5 is an example of a table for storing trace information. The trace information table 500 stores the trace ID 501 and the trace information obtained from the trace record in association with each other.

[0091] Trace ID 501 is an ID that uniquely identifies the trace information and is the same as the ID displayed in Trace ID 401. The trace information is a record of the user's operation of the imaging device 100 over a predetermined period or trajectory point information set by the user, and is referenced during trace playback. Trajectory point number 502 represents the number of the trajectory point added to the trace information. Elapsed time 503 represents the elapsed time from the start of the trace to the point in time when the trajectory point is reached. Pan information 504, tilt information 505, and zoom information 506 describe the pan, tilt, and zoom information for each trajectory point, respectively. Pan position information 507, tilt position information 509, and zoom position information 511 describe the target position of the trajectory point. Pan control point information 508, tilt control point information 510, and zoom control point information 512 describe the parameters for calculating the trajectory line (or control points that constitute the trajectory line).

[0092] Specifically, the trajectory calculation unit 226 calculates trajectory lines connecting the points of each trajectory from the information of the trace information described later. In this embodiment, the time-pan trajectory line, which represents the relationship between time and pan, and the time-tilt trajectory line, which represents the relationship between time and tilt, are each calculated using cubic Bézier curves. In this embodiment, cubic Bézier curves are used to calculate the trajectory lines, but other trajectories that can represent the transitions of time, pan, and tilt may be used, such as cubic spline curves or Lagrangian interpolation polynomials.

[0093] The formula for calculating a cubic Bézier curve can be expressed as follows, using the starting point P0, ending point P1, control point B0, control point B1, and parameter t:

[0094]

number

[0095] Here, the coordinates of the start and end points are the elapsed time and target position of the trajectory points, respectively, and the coordinates of the control points are the values ​​listed in the pan control point information 508, tilt control point information 510, and zoom control point information 512 of the trace information in Figure 5. For example, when calculating the time-pan trajectory line from trajectory point number 1 to trajectory point number 2 in the trace information of Figure 6, the start point P0 is (0.0, -0.83) and the end point P1 is (3.0, -1.3). Also, the control points are B0 (1.0, -0.8) and B1 (2.0, -4.3). The pan-tilt trajectory line, which represents the relationship between pan and tilt, is calculated based on the time-pan trajectory line and the time-tilt trajectory line.

[0096] The method for calculating the trajectory line will be described later. In this embodiment, trace information is recorded for each line, linked to time information. In this example, when the user presses the trace playback button 404 in Figure 4, the pan moves almost stationary while the tilt moves downward from trajectory point number 1 to trajectory point number 2 in Figure 5, as drawn on the pan-tilt graph 410 in Figure 4. From trajectory point number 3 to trajectory point number 6, the pan and tilt move in a counterclockwise circular motion. The zoom moves towards TELE from trajectory point number 1 to trajectory point number 3, and returns to WIDE at trajectory point number 6.

[0097] Furthermore, the trace information according to this embodiment is not limited to parameters related to pan value and tilt value; other parameters may also be recorded. For example, white balance parameters, exposure value parameters, and image quality parameters may also be recorded. Similarly, zoom value parameters do not need to be recorded in this embodiment. Only pan value and tilt value parameters may be recorded as trace information. By recording only pan value and tilt value parameters as trace information in this way, this embodiment can also be applied to a pan / tilt head connected to the imaging device 100.

[0098] (Control flow when editing trace information) Here, the processing flow of the system control unit 204 in the first embodiment will be described with reference to Figures 6, 7, 8, and 9. Figure 6 is a flowchart showing the processing flow for creating and editing a trace on one of the pan-tilt graphs 410, time-pan graph 420, or time-tilt graph 430 displayed on the display unit 201. This flowchart starts when the user operates the GUI displayed on the display unit 201 via the input unit 202 and ends when the user's instructions for creating and editing the trace are drawn on the GUI. Figures 7 and 8 show examples of the GUI display on the display unit 201 when this processing is performed. Figure 7 is a setting screen for creating, editing, and playing back trace information displayed on the display unit 201 of the information processing device 200. The GUI 400 includes a trace ID 401 that indicates identification information of the trace information to be recorded. Furthermore, it includes a recording start button 402 for instructing the start of trace recording, a recording stop button 403 for instructing the stop of trace recording, and a playback button 404 for instructing playback and stopping of trace recording. Furthermore, it includes a point addition button 405, a point editing button 406, and a point deletion button 407 for switching the trace editing state. In addition, it includes a two-dimensional pan-tilt graph 410 showing the relationship between pan value and tilt value, a two-dimensional time-pan graph 420 showing the relationship between pan value and elapsed time, and a two-dimensional time-tilt graph 430 showing the relationship between tilt value and elapsed time. Furthermore, the pan-tilt graph 410 has been updated with the pre-edited trajectory point 711 and the edited trajectory point 712. Figure 8 is Figure 7 with the addition of the pre-modified trajectory points 721 and 731, the pre-modified trajectory lines 713, 723, and 733, the modified trajectory points 722 and 732, and the modified trajectory lines 724 and 734. Figure 9 shows how the control point positions change when the user adds a trajectory point. Details of Figure 9 will be described later.

[0099] In step S601, the user request acquisition unit 223 acquires the operations performed by the user on the GUI displayed on the display unit 201. Depending on whether the point add button 405, point edit button 406, or point delete button 407 is pressed, the acquired request information is output to the trace editing unit 224 and the process proceeds to step S602. For example, in the GUI display example in Figure 7, an example is shown where the point edit button 406 is pressed. When a user operation performed with the point edit button 406 pressed is acquired, the user request acquisition unit 223 outputs information on which graph the user operation was performed on, and the "point edit" information of the "trace editing request" to the trace editing unit 224.

[0100] In step S602, the trace editing unit 224 updates the trace information according to the entered graph information and request information. If the request information is "add point", the unit may request the user to input additional information that cannot be obtained from the graph that the user has operated on, or it may perform supplementation.

[0101] When requesting input from the user, the trace editing unit 224 outputs a command to the trace information drawing unit 227 requesting input, and the trace information drawing unit 227 controls (display control) the display unit 201 to display a GUI requesting input from the user. When interpolating, for example, when a point is added on the pan-tilt graph 410, information about a trajectory point with a target pan and tilt position corresponding to the position specified on the pan-tilt graph 410 is added to the end of the trace information. The elapsed time of the added trajectory point information is set to the elapsed time of the immediately preceding point plus a predetermined time. The control point of the added trajectory point information is left blank, and the control point of the previous point is determined by piecewise cubic Hermitian interpolation using the previous two or three points. Note that other interpolation methods are also acceptable as long as they can obtain the information necessary to smoothly connect the added trajectory point with the preceding and succeeding trajectory points, such as cubic spline interpolation or Akima interpolation.

[0102] Furthermore, if a user operation indicating "add point" is detected on the time-pangraph 420, a trajectory point with the elapsed time and pan target position corresponding to the position specified on the time-pangraph 420 is added to the trace information. If the specified elapsed time is greater than that of the last trajectory point registered in the trace information, a trajectory point is added to the end of the trace information. In this case, the tilt target position is set to the tilt target position of the immediately preceding point. The control point of the added trajectory point information is left blank, and the control point of the previous trajectory point information is determined by piecewise cubic Hermitian interpolation using the previous two or three points. If the specified elapsed time is between any of the trajectory points registered in the trace information, a trajectory point is added to the trace information at the position between them. In this case, the tilt target position is set to the position on the time-tilt trajectory line at the specified elapsed time. The calculation method for the control point values ​​of the added trajectory point information and the previous trajectory point information is explained using Figure 9.

[0103] Figure 9 shows how a new trajectory point is added between two existing trajectory points in the time-pangraph 420. Trajectory points 900, 910, 920, 930, and 940 are trajectory points already registered in the trace information, trajectory point 950 is a trajectory point newly added by the user, and control point 912 is control point B1 of the trajectory point information for trajectory point 910. Control points 921 and 923 are control point B0 of trajectory point information for trajectory point 920 before and after the change, respectively, and control points 922 and 924 are control point B1 of trajectory point information for trajectory point 920 before and after the change, respectively. Control point 931 is control point B0 of trajectory point 930, and control points 951 and 952 are control point B0 and control point B1 of the trajectory point information for the added trajectory point 950, respectively. This section explains how to calculate the control points 951 and 952 of the newly added trajectory point information, and the control points 923 and 924 of the trajectory point information for the previous trajectory point 920.

[0104] First, control points 924 and 951 are determined by piecewise cubic Hermitian interpolation using the existing preceding and succeeding trajectory points 920 and 930, and the added trajectory point 950. For control point 923, the value of the time axis component is first calculated to satisfy the following equation. In this equation, the ratio of the time from trajectory point 920 to control point 921 (T2) to the time from trajectory point 920 to trajectory point 930 (T1) and the ratio of the time from trajectory point 920 to control point 923 (T4) to the time from trajectory point 920 to trajectory point 950 (T3) are kept constant. In the following equation, P0 represents the previous trajectory point 920, P1 represents the next trajectory point 930, P2 represents the added trajectory point 950, B0 represents control point 921, and C0 represents control point 923.

[0105] (1) (B0t-P0t) / (P1t-P0t)=(C0t-P0t) / (P2t-P0t) Afterward, the values ​​for the pan and tilt axis components are calculated so that the slope of the tangent line at the previous trajectory point 920 is the same as before.

[0106] For control point 952, first calculate the value of the time-axis component so that it satisfies the following equation. In the following equation, P0 represents the previous trajectory point 920, P1 represents the next trajectory point 930, P2 represents the added trajectory point 950, B1 represents control point 922, and C1 represents control point 952.

[0107] (2) (P1t-B1t) / (P1t-P0t)=(P1t-C1t) / (P1t-P2t) Then, the values ​​for the pan and tilt axis components are calculated so that the slope of the tangent line at the next trajectory point 930 is the same as the original.

[0108] Furthermore, if points are added on the time-tilt graph 530, trajectory points are added in the same way as when points are added on the time-pan graph 520.

[0109] If the requested information is "point editing," the trace information is modified at the position specified by the user on the graph. For example, if a user operation indicating "point editing" is obtained on the pan-tilt graph 410, the positions of the pan-tilt axis components of the pan and tilt trajectory points and control points of the corresponding trajectory points in the trace information are changed according to the position specified on the pan-tilt graph 410. When a trajectory point is changed, the pan-tilt axis components of the control points are also translated, so the positions of the pan-tilt axis components of the control point B0 of that trajectory point and the control point B1 of the previous trajectory point are changed by the same amount as the change. If the position of the pan-tilt axis component of the pan and tilt control point B0 is changed, the control point B1 of the previous point may also be changed in conjunction to make the movement at the trajectory point smoother. When changing in conjunction, the value of the time axis component of the control point B1 of the previous point should be the same. Furthermore, the pan-tilt axis components of the previous point's control point B1 are recalculated so that the modified control point B0, the trajectory point, and the previous point's control point B1 are aligned in a straight line on the time-pangraph 420 and time-tiltgraph 430. If the position of the pan-tilt axis components of the pan-tilt control point B1 is changed, the control point B0 of the next point may also be changed in conjunction so that the movement at the trajectory point is smooth. If changed in conjunction, the value of the time axis component of the next point's control point B0 should be the same. The pan-tilt axis components of the next point's control point B0 are recalculated so that the modified control point B1, the next point's trajectory point, and the next point's control point B0 are aligned in a straight line on the time-pangraph 420 and time-tiltgraph 430.

[0110] Furthermore, if a user operation indicating "point editing" is detected on the time-pangraph 420, the target position, elapsed time, and control point positions of the corresponding trajectory point in the trace information are changed. When changing the target position, the pan axis component positions of control point B0 of the edited trajectory point information and control point B1 of the previous trajectory point are changed by the same amount as the change. When changing the elapsed time, first, the time axis components of control points B0 and B1 of the edited trajectory point information and control points B0 and B1 of the previous trajectory point information are calculated and changed to satisfy the aforementioned equations (1) and (2). Then, for the pan-tilt axis component values ​​of control points B0 and B1 of the edited trajectory point information and the previous trajectory point information, values ​​are calculated according to the changed time axis component values ​​of control points B0 and B1 at each trajectory point, so as to maintain the slope of the tangent at each trajectory point. Furthermore, when changing control points, if the position of the pan-tilt axis component is changed, the control points of the preceding and succeeding points may also be changed in conjunction, similar to when changing on the pan-tilt graph 410. Furthermore, when a user operation indicating "point editing" is obtained on the time-tilt graph 430, the trace information is changed in the same way as when a user operation indicating "point editing" is obtained on the time-pan graph 520. Note that if there is no previous or next trajectory point, only the information of that trajectory point is changed. Also, if there is no control point, it is not changed.

[0111] If the request information is "point deletion," the information of the trajectory point specified by the user on the graph is deleted from the trace information. In this case, if the last trajectory point is deleted, the control point of the new last trajectory point is deleted. If the space between trajectory points is deleted, the control point of the trajectory point preceding the deleted trajectory point is recalculated. The recalculation method is the same as the method described above for recalculating the control point value when the elapsed time of a trajectory point changes on the time-pangraph 520, but the trajectory point whose elapsed time has changed is interpreted as the trajectory point following the trajectory point that was deleted.

[0112] As described above, after performing control in response to the user's request information, the process proceeds to step S603. In the GUI display example in Figure 7, the trajectory point 711 on the pan-tilt graph 410 has been changed to trajectory point 712, so the target pan and tilt positions of the trace information for the corresponding trajectory point are changed according to the graph's coordinates.

[0113] In step S603, the track line calculation unit 226 calculates the track line for the edited trace information and its surroundings. If any of the pan-related values ​​in the trace information have been changed, the time-pan track line between that track point and the next track point is calculated. If any of the tilt-related values ​​have been changed, the time-tilt track line between that track point and the next track point is calculated. Note that for track points where no control point value exists, the track line between that point and the next track point is not calculated. If there are 0 track points connected to the modified track point, no track line exists, and the process proceeds directly to step S604. Finally, in the section where either the time-pan track line or the time-tilt track line has been calculated, the pan-tilt track line is calculated based on the time-pan track line and the time-tilt track line, and the process proceeds to step S604.

[0114] In step S604, the trace information drawing unit 227 draws the trace information updated in step S602 and the trajectory line calculated in step S603 on the three graphs displayed on the display unit 201 (display control). The pan-tilt graph 410 draws the pan-tilt trajectory line and the pan and tilt coordinates at each trajectory point. The time-pan graph 420 draws the time-pan trajectory line and the time and pan coordinates at each trajectory point, and the time-tilt graph 430 draws the time-tilt trajectory line and the time and tilt coordinates at each trajectory point (display control).

[0115] Figure 8 shows how, in accordance with the user moving the position of trajectory point 711 to trajectory point 712 on the pan-tilt graph 410, the positions of the trajectory points on the time-pan graph 420 and time-tilt graph 430 move from 721 to 722 and from 731 to 732, respectively. In addition, in the three graphs, the trajectory lines in the sections before and after the changed trajectory point are redrawn from 713 to 714, 723 to 724, and 733 to 734, respectively, in accordance with the movement of the trajectory point. At this time, in order to make the correspondence between each trajectory point and trajectory line easier to understand, the marker color and shape may be changed for each trajectory point number, or the trajectory line color and line type may be changed for each trajectory point number.

[0116] Furthermore, to make it easier to understand the changes in the movement of the imaging device when editing trajectory point information, the display of the trajectory line within a predetermined range may be changed. Alternatively, the color of the trajectory line connecting the trajectory points before and after the selected trajectory point may be changed depending on the control item in each section, such as sections where only pan is driven or sections where both pan and tilt are driven. For example, in the pan-tilt graph 410, if the trajectory line connecting the trajectory point before the selected trajectory point is displayed in the color indicating that only pan is driven, it becomes easier to understand that changing the control point position of the selected trajectory point will change the tilt stop state in the previous section.

[0117] Furthermore, if there are trajectory points with the same pan and tilt position in the trace information, they may be displayed differently. For example, the marker size for points with slower elapsed time may be made smaller, or the marker may be displayed in two halves (left and right) or one-third (split) so that all trajectory points at the same position can be viewed simultaneously. Alternatively, the relative positions of each marker and the trajectory line may be swapped at intervals of a few seconds (displayed in a predetermined order). In addition, when the trace information becomes long, it may be possible to select an arbitrary range of trajectory points from the trace information and draw only those points within that selected range on the graph.

[0118] The drawing range can be set automatically. For example, one method is to draw the area before and after the last added or selected trajectory point for a predetermined amount of time or for the number of trajectory points, or to draw the area before or after a trajectory point at the same pan and tilt position (setting the display range).

[0119] As described above, when an instruction to create or edit a trace on a predetermined graph is received from the GUI, the changes to the instructed trace information can be simultaneously displayed on the pan-tilt graph, time-pan graph, and time-tilt graph, respectively. In the first embodiment, an embodiment that controls only pan and tilt was described, but the system is not limited to this and may also control zoom.

[0120] (Second embodiment) In the first embodiment, when an instruction to create and edit trace information on a predetermined graph is received, an example is shown in which the changes to the instructed trace information are simultaneously displayed on the pan-tilt graph, time-pan graph, and time-tilt graph, respectively. In the second embodiment, the control of the trace information drawing unit 227 when trajectory lines overlap on the pan-tilt graph 410 will be explained using Figures 10 and 11.

[0121] For example, when tracing a track cycling or other track racing event, there are situations where you want to repeat a specific pan and tilt trajectory during trace playback. Furthermore, when repeatedly tracing the same trajectory, there are times when you want to speed up the movement between trajectory points for specific actions.

[0122] In the control of the first embodiment, when creating and editing such trace information, the trajectory lines appear to overlap on the pan-tilt graph 410, making it difficult for the user to create and edit. Therefore, in this embodiment, the overlapping parts of the trajectory lines are detected, and the trace information drawing unit 227 changes the drawing of the overlapping parts.

[0123] The control from step S601 to step S604 is the same as in Figure 6, so we will omit the explanation.

[0124] In step S1001, the trace information drawing unit 227 refers to the trace information updated in step S602 and determines whether to change the drawing. In Figure 11, the pan value, tilt value, and zoom value of trajectory point number 1 and trajectory point number 11 are similar. Furthermore, the pan value, tilt value, and zoom value of trajectory point number 2 and trajectory point number 11 are similar. In this embodiment, if the pan value and tilt value of the trajectory points match within a preset range, it is determined that the trajectory points match. Furthermore, if multiple consecutive trajectory points match, the trace information drawing unit 227 determines to change the drawing of the trajectory line between the determined control points. If it is determined in step S1001 that the trajectory points match, the system proceeds to step S1002. If it is not determined in step S1001 that the trajectory points match, the system proceeds to step S604. At this time, the range in which trajectory points are determined to match and the number of consecutive trajectory points may be set by the user, or the determination may be made based on the initial settings. Alternatively, the user may select a parameter indicating the degree of judgment, and the trace information drawing unit 227 may make a judgment based on the user's selection.

[0125] In step S1002, the trace information drawing unit 227 controls the drawing of the trace information based on the trajectory line created in step S603 and the result determined in step S1001, and then proceeds to step S604.

[0126] At this time, the control may be set to change the drawing only on the pan-tilt graph 410, or the control may be set to change the drawing of the same trajectory range on the time-pan graph 420 and time-tilt graph 430.

[0127] Examples of drawing modifications include changing the color of the trajectory line consisting of trajectory points within a related range, but this is not limited to this. For example, the shapes of the markers for the two points before and after a trajectory point that has been determined to be a match may be made different, or the style of the trajectory line (dotted line, dashed line) may be made different. In addition, the transparency of the trajectory points and trajectory lines may be changed to make overlapping parts easier to see.

[0128] As described above, in this embodiment, the trace information drawing unit 227 detects overlapping sections of the trajectory lines and modifies the drawing of the overlapping sections, allowing the user to identify the trace information of the section to be edited while performing trace editing.

[0129] (Third embodiment) In the first embodiment, when an instruction to create and edit trace information on a predetermined graph is received, an example is shown in which the changes to the instructed trace information are simultaneously displayed on the pan-tilt graph, time-pan graph, and time-tilt graph, respectively. In the third embodiment, the process of displaying the captured image when creating and editing trace information created on the GUI is further described.

[0130] For example, when correcting trace information, it is not possible to check what kind of image will be acquired at the corrected pan, tilt, and zoom positions.

[0131] Therefore, in this embodiment, trace information can be edited while checking the captured image.

[0132] In the third embodiment, the process of driving the imaging device when editing a trace on the GUI will be described. The process of the information processing device 200 according to the third embodiment will be described below with reference to Figures 12, 13, and 14.

[0133] Figure 12 shows an example of a GUI displayed on the display unit 201 of the information processing device 200. The GUI shown in Figure 12 includes, in addition to the configuration of the GUI displayed on the display unit 201 shown in Figure 4, a camera address input unit 1201 and a camera image display unit 1202.

[0134] The camera address input unit 1201 is an interface for inputting the IP address of the camera targeted by the camera information acquisition unit 220, the image acquisition unit 221, and the PTZ control unit 222. When an IP address is input, the image acquisition unit 221 displays the image acquired from that camera on the camera image display unit 1202. In this embodiment, an IP address is input, but this is not the only option. For example, the connected imaging device could be selected from a drop-down list.

[0135] The camera image display unit 1202 is for displaying video from the imaging device 100 acquired by the image acquisition unit 221 based on the IP address input to the camera address input unit 1201. In this embodiment, the information processing device 200 will continue to display the images captured by the imaging device 100 on the display unit 202 while connected to the imaging device 100, but is not limited to this. For example, it may display the images when a user performs a user operation to request image display. Alternatively, the captured images may be displayed after controlling the pan, tilt, and zoom values ​​to correspond to the trajectory points to be edited and added. In other words, the image being driven may not be displayed on the display unit 202, and only the captured images with the pan, tilt, and zoom values ​​corresponding to the trajectory points may be displayed.

[0136] Figures 13 and 14 are flowcharts illustrating the process of editing (or creating) trace information on one of the pan-tilt graphs 410, time-pan graph 420, or time-tilt graph 430 displayed on the display unit 201. Figure 13 is a flowchart for adding a new trajectory point to the trace information when creating or editing trace information. Figure 14 is a flowchart for editing the information of a trajectory point already registered in the trace information when editing a trace. This flowchart is started by the user operating the GUI displayed on the display unit 201 via the input unit 202 and ends when the user updates the trace information.

[0137] First, we will explain the process of adding a new trajectory point to the trace information using the flowchart in Figure 13.

[0138] Step S601 is the same as the configuration and content described in Figure 6, so its explanation will be omitted.

[0139] In step S1301, the user request acquisition unit 223 determines whether the acquired request information is "add point". In this embodiment, it is determined to be "add point" when the user performs a user operation via the input unit 202 while the add point 405 button is pressed in the GUI of Figure 12, but it is not limited to this, and it is sufficient if "add point" is instructed by a user operation. For example, this could be when "add point" is instructed by a voice command, when "add point" is instructed using a gesture recognition system, when a specific swipe operation is performed on a touchscreen, when a specific shortcut key is pressed on the keyboard, when "add point" is selected from the right-click menu of the mouse, or when a specific icon is tapped in a mobile application. If the user request acquisition unit 223 determines that the acquired request information is "add point", it proceeds to step S602, and if it does not determine that the acquired request information is "add point", it updates the trace information and terminates this flowchart.

[0140] Step S602 is the same as the configuration and content explained in Figure 6, so its explanation will be omitted.

[0141] In step S1302, the PTZ control unit 222 converts the trajectory point information acquired in step S601 into control information for driving the imaging device 100 and outputs it to the imaging device 100. Here, the control information for driving the imaging device 100 includes, for example, information on the target positions for pan, tilt, and zoom, as well as information on driving time and driving speed. As a result, the camera image display unit 1202 displays the camera image captured by the imaging device 100 at the added trajectory point.

[0142] Steps S603 and S604 are the same as those described in Figure 6, so their explanation will be omitted.

[0143] As explained above, when an instruction to add a point is received from the GUI as part of the trace editing method, the imaging device can be driven to the specified PTZ position.

[0144] Next, we will explain the process of editing the information of track points already registered in the trace information using the flowchart in Figure 14. This flowchart starts when the user operates the GUI displayed on the display unit 201 via the input unit 202 and ends when the user updates the trace information.

[0145] Step S601 is the same as the configuration and content described in Figure 6, so its explanation will be omitted.

[0146] In step S1401, the user request acquisition unit 223 determines whether the acquired request information is for "point editing" or "point deletion". In this embodiment, it is determined to be "point editing" when the point editing 406 button is pressed in the GUI of Figure 12 and the user performs a user operation via the input unit 202, but it is not limited to this, and it is sufficient if "point editing" is instructed by a user operation. For example, this could be when "point editing" is instructed by a voice command, when "point editing" is instructed using a gesture recognition system, when a specific swipe operation is performed on a touchscreen, when a specific shortcut key is pressed on the keyboard, when "point editing" is selected from the right-click menu of the mouse, or when a specific icon is tapped in a mobile application.

[0147] Similarly, in this embodiment, the dot erase button 407 in the GUI of Figure 12 is pressed, and the system determines that it is a "dot erase" operation when the user performs a user operation via the input unit 202. However, it is not limited to this; any operation that instructs "dot erase" is acceptable. The user request acquisition unit 223 proceeds to step S1402 if it determines that the acquired request information is either a "dot edit" or a "dot erase." If the acquired request information is not determined to be either a "dot edit" or a "dot erase," the trace information is updated, and this flowchart is terminated.

[0148] In step S1402, the PTZ control unit 222 converts information on at least one of the pan, tilt, and zoom values ​​corresponding to the user operation acquired in step S601 into control information for driving the imaging device 100 and outputs it to the imaging device 100. Here, the control information for driving the imaging device 100 includes, for example, pan, tilt, and zoom position information, as well as driving time and driving speed information. As a result, the camera image display unit 1202 displays the camera image projected by the imaging device 100 at the specified trajectory point. After converting information on at least one of the pan, tilt, and zoom values ​​specified by the user into control information for driving the imaging device 100 and outputting it to the imaging device 100 in step S1402, the process transitions to step S1403.

[0149] In step S1403, the user request acquisition unit 223 determines whether the acquired request information is a "point deletion". The determination method is the same as in step S1401, so the explanation is omitted. If the user request acquisition unit 223 determines that the acquired request information is a "point deletion", it proceeds to step S602; otherwise, it proceeds to step S1404.

[0150] In step S1404, the trace editing unit 224 determines whether the operation performed by the user is an edit of trace information. If an instruction is given to change the position of trajectory points or control points by an operation such as dragging (YES in step S1404), the PTZ position of the imaging device 100 corresponding to the edited trajectory points or control points is output to the PTZ control unit 222 and the process proceeds to S1405. However, if the trace information is edited in a way that is not related to the PTZ position of the imaging device 100, such as editing the elapsed time or white balance, the PTZ position does not need to be output to the PTZ control unit 222. If the trace editing unit 224 determines that the operation performed by the user is an edit of trace information, the process proceeds to step S1405. If the operation performed by the user is not determined to be an edit of trace information (for example, adding a point or driving the imaging device 100 to a position corresponding to a position specified on the graph to check the camera image), the flowchart ends. At this time, if the trace information is not edited within a predetermined time period after acquiring the graph operation details in step S601, it may be determined that it is not an edit of trace information. Furthermore, if the trace information is not edited within a predetermined time period after outputting a control command to the imaging device 100 in step S1402, it may be determined that the trace information is not being edited.

[0151] Step S1405 is the same configuration and content as described in step S602 in Figure 6, so its explanation is omitted. In step S1405, the trace editing unit 224 updates the trace information according to the input graph information and request information, and then proceeds to step S1406. In step S1406, the PTZ control unit 222 sends a control command to the imaging device 100 using the control information input from the trace editing unit 224. Here, the control information for driving the imaging device 100 includes, for example, information on the target position of pan, tilt, and zoom, as well as elapsed time and drive speed information. As a result, the camera image display unit 1202 displays the camera image projected by the imaging device 100 at the added trajectory point. At this time, if the trace information editing content determined in step S1404 is an edit of trace information unrelated to the PTZ position of the imaging device 100, such as editing the elapsed time or editing the white balance, this step does not need to be performed.

[0152] Steps S603 and S604 are the same as those described in Figure 6 in terms of configuration and content, so their explanation will be omitted.

[0153] As described above, when an instruction to edit the trace is received from the GUI, the imaging device can be driven to the specified PTZ position. In the second embodiment, an embodiment that controls only pan and tilt was described, but the system is not limited to this and may also control zoom.

[0154] In addition, in S1402 and S1407, when the operation of the imaging device 100 ends, the GUI displayed on the display unit 201 may be configured to show that the operation has ended (is complete) and the target position has been reached. For example, the color of the reached trajectory point may be superimposed as the frame color of the camera image display unit 1202. Alternatively, when the operation of the imaging device 100 ends, the image acquired by the image acquisition unit 221 may be associated with the trajectory point and stored in the storage unit 203. The stored images may be displayed as a list, or the stored image may be displayed when a trajectory point is selected.

[0155] Furthermore, if the user specifies a PTZ position in S1402 or S1407 where the imaging device cannot be driven, the system may be configured to change it to a drivable position and update the trace information and graph display. For example, if the position of the pan-tilt axis component of the control point is changed and part of the trajectory line falls at the edge of the imaging device's control range, the system may be configured to draw the graph in accordance with the actual movement of the imaging device.

[0156] (Fourth embodiment) In the first embodiment, when an instruction to create and edit trace information on a predetermined graph is received, an example is shown in which the changes to the instructed trace information are simultaneously displayed on the pan-tilt graph, time-pan graph, and time-tilt graph, respectively. In the fourth embodiment, a process is described in which a mark representing the current PTZ position of the imaging device is displayed on each graph when playing back the trace information created on the GUI. The processing of the information processing device 200 according to the fourth embodiment will be described below with reference to Figures 15 and 16.

[0157] For example, when reproducing trace information, it is sometimes necessary to identify areas that need correction while reviewing the captured image.

[0158] In this way, when reproducing trace information, it is difficult to understand which part of the series of trace information is being reproduced using the control method of the first embodiment. Therefore, in this embodiment, the current PTZ position of the imaging device is displayed on each graph so that it can be identified during trace reproduction.

[0159] As described above, displaying the pan, tilt, and zoom positions of the imaging device on a graph showing the movement of the imaging device makes it easier to edit the trace information.

[0160] Figure 15 is a flowchart showing the processing flow of the system control unit 204 when playing back a trace created on a graph displayed on the display unit 201. This flowchart starts when the user presses the play button 404 on the GUI displayed on the display unit 201 via the input unit 202, and ends when the trace playback is finished or when the user inputs a command to stop the trace. Figure 16 shows an example of the GUI display on the display unit 201 when this process is performed.

[0161] In step S1501, the trace playback unit 225 receives a trace playback request input from the user request acquisition unit 223, reads the trace information stored in the storage unit 203, and sends a PTZ drive instruction to the PTZ control unit 222, instructing the imaging device 100 to perform PTZ driving.

[0162] In step S1502, the camera information acquisition unit 220 acquires the current PTZ position of the imaging device 100 and outputs it to the trace information drawing unit 227.

[0163] In step S1503, the trace information drawing unit 227 draws the current PTZ position of the imaging device 100, which is input from the camera information acquisition unit 220. Furthermore, based on the elapsed time since the start of trace playback, which is input from the trace playback unit 225, it draws a marker representing the PTZ position of the imaging device 100 during trace playback. In other words, it controls the display of the marker on three graphs displayed on the display unit 201: the pan-tilt graph 410, the time-pan graph 420, and the time-tilt graph 430. At this time, the pan-tilt graph 410 also displays a rectangle corresponding to the current zoom size based on the zoom value acquired in step S1502 (Figure 16, rectangle 1601). When zooming in during trace information playback, the rectangle size decreases in accordance with the zoom in. Similarly, when zooming out, the rectangle size increases in accordance with the zoom out. By displaying the rectangle size during trace information playback in this way, the user can easily grasp the zoom value and edit the image.

[0164] In the GUI example in Figure 16, markers 1602 indicating the pan and tilt positions are displayed on the pan-tilt graph 410, time-pan graph 420, and time-tilt graph 430, respectively. Additionally, markers 1603 indicating the time since the start of playback and the pan position, and marker 1604 indicating the time since the start of playback and the tilt position are displayed. Furthermore, on the pan-tilt graph 410, a rectangle 1601 corresponding to the current zoom size is also displayed.

[0165] In S1504, it is determined whether the trace playback by the information processing device 200 has finished. If a "stop playback" request is entered in the user request acquisition unit 223, or if the trace playback has finished up to the end of the trace information (YES in S1504), the flowchart is terminated. Otherwise (NO in S1504), the process returns to S1501.

[0166] As explained above, when playing back traces created on the GUI, a mark representing the current PTZ position of the imaging device can be displayed on each graph.

[0167] Furthermore, when the information processing device 200 is playing back the trace, the color of the frame of the camera image display unit 1202 displayed on the display unit 201 may be changed to match the color of the trajectory line of the section of the trace information being played back, making the currently played section easier to understand. Alternatively, when the user-specified trajectory point is passed during trace playback, the frame of the camera image display unit 1202 displayed on the display unit 201 may be made to blink or otherwise highlight the passage of the trajectory point.

[0168] Furthermore, a plot button is provided on the GUI displayed on the display unit 201, and when this button is pressed during trace playback on the information processing device 200, the pan, tilt, and zoom position information at that time is stored. In addition, plots may be drawn on the pan-tilt graph 410, time-pan graph 420, and time-tilt graph 430, respectively, so that areas of interest during trace playback can be saved. At this time, the image captured by the imaging device acquired by the image acquisition unit 221 may also be stored.

[0169] Furthermore, in this embodiment, the rectangle 1601 corresponding to the zoom value may also be displayed when editing trace information. By dragging the rectangle 1601 or specifying the rectangle size, the user can display the pan, tilt, and zoom positions of the imaging device on the graph showing the movement of the imaging device as described above, making it easier to edit trace information.

[0170] In this embodiment, when playing back trace information, the current PTZ position of the imaging device 100 is obtained from the imaging device 100. Furthermore, markers 1602 indicating the pan and tilt positions are drawn on the pan-tilt graph 410, time-pan graph 420, and time-tilt graph 430, respectively, but this is not limited to this. For example, the current position of the imaging device 100 is obtained at predetermined intervals while the imaging device 100 and the information processing device 200 are connected in a communicative manner. Furthermore, markers 1602 indicating the pan and tilt positions may be drawn on the pan-tilt graph 410, time-pan graph 420, and time-tilt graph 430, respectively. In this case, it is easy to see how far the current PTZ position is from the PTZ position indicated by the trace information, and it is easy to be aware of where to edit to obtain an arbitrary captured image when playing back the trace. Also, when the state becomes ready for playback or editing of the trace, the current position of the imaging device 100 is obtained. Furthermore, markers 1602 indicating the pan and tilt positions may be drawn on the pan-tilt graph 410, time-pan graph 420, and time-tilt graph 430, respectively. In this case, even if communication delays occur, the current PTZ position can be drawn before trace playback, making it immediately clear which position needs to be edited during trace playback.

[0171] [Other embodiments] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0172] Embodiments of this disclosure include the following configurations, methods, and programs.

[0173] [Configuration 1] An acquisition means for acquiring captured images from an imaging device capable of panning and tilting, A display control means controls the display of a first information display area that displays information on the transition of the pan and tilt positions of the imaging device, a second information display area that shows at least one of the time transitions of the pan position and the time transitions of the tilt position, and a third information display area that displays the image captured by the acquisition means. An editing means for editing the information displayed in the first information display area and the second information display area, The imaging device has control means for controlling the pan-tilt position, The information processing apparatus is characterized in that the control means controls the first information display area or the second information display area to be driven to a pan-tilt position specified by the user.

[0174] [Configuration 2] The information processing apparatus according to Configuration 1, characterized in that, when the control means is capable of controlling the pan-tilt position, the display control means controls the captured image acquired by the acquisition means to be displayed in the third information display area.

[0175] [Configuration 3] The information processing apparatus according to any one of configurations 1 to 2, characterized in that while the imaging device is in operation, the display control means controls the imaging image acquired by the acquisition means to be displayed in the third information display area.

[0176] [Structure 4] The imaging device is capable of zooming. The control means controls the zoom position, The information processing apparatus according to any one of configurations 1 to 3, characterized in that the control means controls the first information display area or the second information display area to be driven to a pan, tilt, and zoom position specified by the user.

[0177] [Composition 5] The pan-tilt position transition information and the transition speed of the pan or tilt position are trace information that stores the transitions of multiple operations on the imaging device. An information processing device according to any one of configurations 1 to 4 characterized by the above.

[0178] [Composition 6] The first information display area is a two-dimensional graph with pan as the first axis and tilt as the second axis, and the second information display area includes at least one of the following: a two-dimensional graph with time as the first axis and pan as the second axis, and a two-dimensional graph with time as the first axis and tilt as the second axis. The aforementioned trace information displays the control information of the imaging device as a trajectory line. An information processing device according to any one of configurations 1 to 5 characterized by the above.

[0179] [Composition 7] The editing means includes adding at least one trajectory point from among a plurality of trajectory points constituting the trajectory line, deleting the trajectory point, changing the pan or tilt position corresponding to the trajectory point, changing the arrival time of the trajectory point, and changing the trajectory line. An information processing device according to any one of configurations 1 to 6 characterized by the above.

[0180] [Structure 8] The aforementioned trajectory points have identification information that allows for the identification of each trajectory point. The aforementioned identification information includes the target arrival position and target arrival time for the pan position and tilt position. An information processing device according to any one of configurations 1 to 7 characterized by the above.

[0181] [Composition 9] The system includes a calculation means for calculating a trajectory line based on the aforementioned trajectory points, An information processing apparatus according to any one of configurations 1 to 8, characterized in that the control means controls the imaging device based on the trajectory line calculated by the calculation means.

[0182] [Configuration 10] When adding a trajectory point to the trajectory line displayed in the first information display area in the editing means, a predetermined time is added to the time of the trajectory point immediately preceding the trajectory point and recorded as identification information for the trajectory point; when adding a trajectory point to the trajectory line displayed in the second information display area, a predetermined pan or tilt value is recorded as identification information for the trajectory point. An information processing device according to any one of configurations 1 to 9, characterized in that the control means controls the imaging device according to the pan-tilt position of the trajectory points added by the trajectory point adding means.

[0183] [Composition 11] The information processing apparatus according to any one of configurations 1 to 10, characterized in that when the editing means edits the identification information of the trajectory points of the trajectory line, the control means controls the imaging device according to the pan-tilt position based on the edited identification information.

[0184] [Composition 12] The information processing device according to any one of configurations 1 to 11, characterized in that when a predetermined position in the first information display area or the second information display area is specified, the control means controls the imaging device according to the pan-tilt position corresponding to the predetermined position.

[0185] [Composition 13] It has an information acquisition means for acquiring control information from the imaging device, The control means controls the imaging device, and when information indicating that control has been completed is obtained from the imaging device, the storage means saves the image acquired by the acquisition means in association with the trajectory points. An information processing device according to any one of configurations 1 to 12, characterized by the above.

[0186] [Composition 14] When a trajectory point is selected by the editing means, the display control means controls the display to show the captured image stored in association with the trajectory point. An information processing device according to any one of configurations 1 to 13 characterized by the above.

[0187] [Method 1] An acquisition step in which an image is acquired from an imaging device capable of pan and tilt movement, A display control step that controls the display of a first information display area that displays information on the transition of the pan and tilt positions of the imaging device, a second information display area that shows at least one of the time transitions of the pan position and the time transitions of the tilt position, and a third information display area that displays the image captured by the acquisition means. An editing step for editing the information displayed in the first information display area and the second information display area, The system includes a control step for controlling the pan-tilt position of the imaging device, A control method for an information processing device, characterized in that, in the control step, the device is controlled to drive either the first information display area or the second information display area to a pan-tilt position specified by the user.

[0188] [program] A program that causes the computer of the information processing device to execute each step described in Method 1. [Explanation of Symbols]

[0189] 100 Imaging device 101 Imaging Unit 102 Lens drive unit 103 Pan drive unit 104 Tilt drive unit 105 Image Processing Unit 107 Pan-Tilt Control Unit 108 System Control Unit 109 Storage section 110 Communications Department 200 Information Processing Devices 201 Display section 202 Input Section 203 Storage section 204 System Control Unit 205 Communications Department

Claims

1. An acquisition means for acquiring captured images from an imaging device capable of panning and tilting, A display control means controls the display of a first information display area that displays information on the transition of the pan and tilt positions of the imaging device, a second information display area that shows at least one of the time transitions of the pan position and the time transitions of the tilt position, and a third information display area that displays the image captured by the acquisition means. An editing means for editing the information displayed in the first information display area and the second information display area, The imaging device has control means for controlling the pan and tilt position, The information processing apparatus is characterized in that the control means controls the first information display area or the second information display area to be driven to a pan-tilt position specified by the user.

2. The information processing apparatus according to claim 1, characterized in that, when the control means is capable of controlling the pan-tilt position, the display control means controls the captured image acquired by the acquisition means to be displayed in the third information display area.

3. The information processing apparatus according to claim 1, characterized in that, while the imaging device is in operation, the display control means controls the imaging image acquired by the acquisition means to be displayed in the third information display area.

4. The imaging device is capable of zooming. The control means controls the zoom position, The information processing apparatus according to claim 1, characterized in that the control means controls the first information display area or the second information display area to be driven to a pan, tilt, or zoom position specified by the user.

5. The pan-tilt position transition information and the time transition of the pan position or tilt position are trace information that stores the transitions of multiple operations on the imaging device. The information processing apparatus according to feature 1.

6. The first information display area is a two-dimensional graph with pan as the first axis and tilt as the second axis, and the second information display area includes at least one of the following: a two-dimensional graph with time as the first axis and pan as the second axis, and a two-dimensional graph with time as the first axis and tilt as the second axis. The aforementioned trace information displays the control information of the imaging device as a trajectory line. The information processing apparatus according to feature 1.

7. The editing means includes adding at least one trajectory point from among a plurality of trajectory points constituting the trajectory line, deleting the trajectory point, changing the pan or tilt position corresponding to the trajectory point, changing the target arrival time of the trajectory point, and changing the trajectory line. The information processing apparatus according to feature 6.

8. The aforementioned trajectory points have identification information that allows for the identification of each trajectory point. The identification information includes the target positions of the pan position and tilt position, and the time to reach the target. The information processing apparatus according to feature 7.

9. The system includes a calculation means for calculating a trajectory line based on the aforementioned trajectory points, The information processing apparatus according to claim 7, characterized in that the control means controls the imaging device based on the trajectory line calculated by the calculation means.

10. When adding a trajectory point to the trajectory line displayed in the first information display area in the editing means, a predetermined time is added to the time of the trajectory point immediately preceding the trajectory point and recorded as identification information for the trajectory point; when adding a trajectory point to the trajectory line displayed in the second information display area, a predetermined pan or tilt value is recorded as identification information for the trajectory point. The information processing apparatus according to claim 9, characterized in that the control means controls the imaging device according to the pan-tilt position of the added trajectory point.

11. The information processing apparatus according to claim 9, characterized in that when the editing means edits the identification information of the trajectory points of the trajectory line, the control means controls the imaging device according to the pan-tilt position based on the edited identification information.

12. The information processing apparatus according to claim 9, characterized in that when a predetermined position in the first information display area or the second information display area is specified, the control means controls the imaging device according to the pan-tilt position corresponding to the predetermined position.

13. It has an information acquisition means for acquiring control information from the imaging device, The control means controls the imaging device, and when information indicating that control has been completed is obtained from the imaging device, the storage means saves the image acquired by the acquisition means in association with the trajectory points. The information processing apparatus according to any one of claims 10 to 12.

14. When a trajectory point is selected by the editing means, the display control means controls the display to show the captured image stored in association with the trajectory point. Information device according to feature 13

15. An acquisition step in which an image is acquired from an imaging device capable of pan and tilt movement, A display control step controls the display of a first information display area that displays information on the transition of the pan and tilt positions of the imaging device, a second information display area that shows at least one of the time transitions of the pan position and the time transitions of the tilt position, and a third information display area that displays the image acquired in the acquisition step. An editing step for editing the information displayed in the first information display area and the second information display area, The system includes a control step for controlling the pan-tilt position of the imaging device, A control method for an information processing device, characterized in that, in the control step, the device is controlled to drive either the first information display area or the second information display area to a pan-tilt position specified by the user.

16. A program for causing a computer to function as one of the means of the information processing apparatus according to any one of claims 1 to 13.

Images