Trace control device, control method and program for trace control device

Abstract

The trace information can be modified, and information indicating the time interval of the modification can be notified during trace playback control. [Solution] The trace control device includes: recording means for recording trace information of the control of the imaging unit in association with time information when a request for recording of trace information of the control of the imaging unit is received; modification means for modifying the trace information recorded by the recording means or the time information when a request for modification of the trace information of the control of the imaging unit is received; playback means for controlling the playback of the trace of the control of the imaging unit based on the recorded or modified trace information and time information when a request for playback of the trace information of the control of the imaging unit is received; and notification means for notifying the user of information indicating the time interval modified by the modification means when a request for modification of the trace information is received before playback control by the playback means.

Description

【Technical Field】 【0001】 The present invention relates to a trace control device, a control method for the trace control device, and a program. 【Background Art】 【0002】 In recent years, with the expansion of the video distribution market, the number of imaging systems that use cameras capable of remote control and video shooting via a network for shooting at weddings and lectures has been increasing. As a general method for operating a camera that can be remotely controlled, a method using a hard controller is used, but a method using an application (soft controller) that operates on a personal computer (PC) or a mobile terminal has also been realized. 【0003】 The camera can be remotely controlled via a graphic user interface (GUI) on the application, but it is also possible to connect hardware such as a joystick to a PC or a mobile terminal and realize a part of the remote operation using the hardware. As remote operations, pan-tilt-zoom (PTZ) operations, preset operations, trace operations, etc. are possible. 【0004】 The trace function is a function that can reproduce camera control based on operation information in which information operated by the user is recorded along the time series. If an incorrect operation is performed when recording the operation, this trace function must be recorded again from the beginning. Therefore, it is necessary to repeat the recording many times until the trace information intended by the user is created. 【0005】 Patent Document 1 discloses a method of realizing highly convenient correction of trace information by displaying trace information acquired from an imaging device in a graph and correcting a locus on the graph. 【0006】 Furthermore, Patent Document 2 discloses a method for conveniently modifying trace information by receiving a camera control request from the user during trace playback and updating the trace information being played back when a control request is received. [Prior art documents] [Patent Documents] 【0007】 [Patent Document 1] Japanese Patent Publication No. 2023-117195 [Patent Document 2] Japanese Patent Publication No. 2023-155733 [Overview of the Initiative] [Problems that the invention aims to solve] 【0008】 However, the technologies disclosed in Patent Document 1 and Patent Document 2 have the problem that it is not possible to determine which period the modifications were made during trace playback after the modifications have been made. 【0009】 For example, if trace information is modified on a graph as in Patent Document 1, it is possible to obtain the operation information captured by the camera, but during trace playback, it is not possible to determine the period during which the modification was made on the video. Also, when the camera control movement speed described in the trace information is modified, if the difference in the modification is small, it may not be possible to determine the period during which the speed was changed on the video. 【0010】 An objective of the present invention is to enable modification of trace information and to provide information indicating the time interval of the modification during trace playback control. [Means for solving the problem] 【0011】 The trace control device includes: recording means for recording trace information of the control of the imaging unit in association with time information when a request for recording of trace information of the control of the imaging unit is received; modification means for modifying the trace information recorded by the recording means or the time information when a request for modification of the trace information of the control of the imaging unit is received; playback means for controlling the playback of the trace of the control of the imaging unit based on the recorded or modified trace information and time information when a request for playback of the trace information of the control of the imaging unit is received; and notification means for notifying the user of information indicating the time interval modified by the modification means when a request for modification of the trace information is received before playback control by the playback means. [Effects of the Invention] 【0012】 According to the present invention, trace information can be modified, and information indicating the time interval of modification can be notified during trace playback control. [Brief explanation of the drawing] 【0013】 [Figure 1] This is a diagram showing an example of the external appearance of an imaging device. [Figure 2] This is a diagram showing an example of an imaging system. [Figure 3] This is a configuration diagram showing an example of an imaging system configuration. [Figure 4] This is a block diagram of a trace control unit according to the first embodiment of the present invention. [Figure 5] This figure shows an example of trace information according to the first embodiment of the present invention. [Figure 6] This figure shows an example of trace information including change information according to the first embodiment of the present invention. [Figure 7] This is a flowchart showing the processing procedure of the trace control unit according to the first embodiment of the present invention. [Figure 8] This is a block diagram of a trace control unit according to a second embodiment of the present invention. [Figure 9]It is a diagram showing an example of a GUI according to a second embodiment of the present invention. [Figure 10] It is a diagram showing an example of specifying a change interval when changing the position information according to a second embodiment of the present invention. [Figure 11] It is a diagram showing an example of specifying a change interval when changing the time information according to a second embodiment of the present invention. [Figure 12] It is a diagram showing an example of specifying a change interval when changing the speed information according to a second embodiment of the present invention. [Figure 13] It is a flowchart showing a processing procedure of a trace control unit according to a second embodiment of the present invention. 【Embodiments for Carrying Out the Invention】 【0014】 (First Embodiment) FIG. 1 is a diagram showing an example of the appearance of an imaging device 100 according to a first embodiment of the present invention. The pan drive mechanism 101 can change the imaging area of the imaging device 100 in the pan direction. The tilt drive mechanism 102 can change the imaging area of the imaging device 100 in the tilt direction. The zoom mechanism 103 changes the angle of view of the imaging device 100. 【0015】 FIG. 2 is a configuration diagram of an imaging system including the imaging device 100. The imaging system includes the imaging device 100, a client device 200, and a network 300. The client device 200 is a client device indicating an external device. The imaging device 100 and the client device 200 are connected to each other in a communicable state via the network 300. 【0016】 The client device 200 performs controls such as angle-of-view control and image-quality control on the imaging device 100. The imaging device 100 transmits responses to those control commands to the client device 200. 【0017】 Referring to FIG. 3, the configurations of each part of the imaging device 100, the network 300, the configurations of each part of the client device 200, and the functions will be described. 【0018】 The imaging device 100 includes an imaging unit 111, an image processing unit 112, a lens drive unit 113, a zoom control unit 114, a pan drive unit 115, a tilt drive unit 116, a pan / tilt control unit 117, a recording unit 118, a control system 119, a communication unit 120, and a video IF control unit 121. 【0019】 The imaging unit 111 converts the subject into an electrical signal by photoelectric conversion. 【0020】 The image processing unit 112 performs predetermined image processing and compression encoding processing on the signal converted photoelectrically by the imaging unit 111 to generate image data. The image processing unit 112 also generates image data with an emphasis frame superimposed upon it, upon receiving a request for emphasis frame superimposition from the control system 119 (described later). Note that the image data generated is not limited to a single image; multiple image data with different resolutions and video qualities may be generated simultaneously. The image processing unit 112 then transmits the generated image data to the control system 119. 【0021】 The lens drive unit 113 consists of a drive system for the focus lens and the zoom lens, and its operation is controlled by the zoom control unit 114. 【0022】 The zoom control unit 114 controls the lens drive unit 113 based on instructions transmitted from the control system 119. The zoom control unit 114 also transmits focus movement information and zoom movement information to the control system 119. 【0023】 The pan drive unit 115 consists of a mechanical drive system that performs panning and a motor that is the drive source, and its operation is controlled by the pan-tilt control unit 117. 【0024】 The tilt drive unit 116 consists of a mechanical drive system and a motor that performs the tilting motion, and its operation is controlled by the pan-tilt control unit 117. 【0025】 The pan-tilt control unit 117 controls the pan drive unit 115 and the tilt drive unit 116 based on the transmitted instructions. 【0026】 The recording unit 118 stores image data, various setting commands, etc. 【0027】 The control system 119 analyzes the transmitted camera control commands and performs processing according to the commands. The control system 119 issues instructions to the image processing unit 112 for image quality adjustment, instructions to the zoom control unit 114 for zoom and focus control, and instructions to the pan / tilt control unit 117 for pan / tilt operation. The control system 119 acquires the image data generated by the image processing unit 112 and transmits it to the communication unit 120 and the video IF control unit 121. 【0028】 The communication unit 120 transmits image data sent from the control system 119 to the client device 200. The communication unit 120 also receives various setting commands and camera control commands sent from the client device 200 and transmits them to the control system 119. Furthermore, the communication unit 120 transmits the response of the imaging device 100 to the commands sent from the client device 200 to the client device 200. 【0029】 The video IF control unit 121 outputs image data transmitted from the control system 119 to various IFs such as HDMI®, 3G-SDI, and 12G-SDI connected to the camera body. The video IF control unit 121 also outputs information about connectable IFs to the control system 119. 【0030】 The client device 200 can be a general-purpose computer such as a personal computer, or a mobile terminal such as a tablet. 【0031】 The client device 200 includes a display unit 201, an input unit 202, a system control unit 203, a communication unit 204, and a recording unit 205. 【0032】 The display unit 201 uses a display device such as an LCD projector or LCD monitor to display images acquired from the imaging device 100 and a graphic user interface (hereinafter referred to as GUI) for camera control. 【0033】 The input unit 202 uses a keyboard, mouse, touch panel, or other pointing device, and the user of the client device 200 operates the GUI via the input unit 202. 【0034】 The system control unit 203 generates various setting commands and camera control commands in response to the user's GUI operations and transmits them to the imaging device 100 via the communication unit 204. The system control unit 203 also receives the responses from the imaging device 100 to the transmitted setting commands and camera control commands via the communication unit 204. The system control unit 203 also displays the image data received from the imaging device 100 via the communication unit 204 on the display unit 201. 【0035】 The communication unit 204 transmits various setting commands and camera control commands sent from the system control unit 203 to the imaging device 100. The communication unit 204 also transmits image data transmitted from the imaging device 100 and the imaging device 100's response to commands transmitted from the client device 200 to the system control unit 203. 【0036】 The recording unit 205 records camera information and other data to internal storage and external storage. 【0037】 In this way, the client device 200 can perform various camera controls of the imaging device 100 via the network 300. 【0038】 The configuration of the first embodiment of the present invention will be described with reference to Figure 4. Figure 4 is a block diagram showing an example of the functional configuration of the trace control unit 400 included in the control system 119 according to this embodiment. 【0039】 The trace control unit 400 interprets the PTZ camera control commands received from the client device 200 and performs trace control. In this embodiment, the trace control unit 400 is described as a function within the control system 119 of the imaging device 100, but it is also acceptable to configure the system control unit 203, which is included in the client device 200, to perform trace control. 【0040】 The trace control unit 400 includes a trace control request receiving unit 401, a trace recording unit 402, a trace editing unit 403, a trace playback unit 404, a control request unit 405, and a change section notification unit 406. 【0041】 The trace control request receiving unit 401 receives trace control requests transmitted from the client device 200. There are three types of trace control requests: "recording requests," "modification requests," and "playback requests." A "recording request" has request information for "start recording" and "stop recording," a "modification request" has request information for "start editing," "confirm editing," "modify," and "reset," and a "playback request" has request information for "start playback" and "stop playback." If the trace control request is a "recording request," the request information is output to the trace recording unit 402. If the trace control request is a "modification request," the request information is output to the trace editing unit 403. If the trace control request is a "playback request," the request information is output to the trace playback unit 404. 【0042】 The trace recording unit 402 records trace information to the recording unit 118 based on the request information input from the trace control request reception unit 401. If the request information is "start recording", it starts recording; if the request information is "stop recording", it stops recording. 【0043】 The trace editing unit 403 reads the trace information recorded in the recording unit 118 from the request information input from the trace control request reception unit 401 and edits the trace information. If the request information is "Start Editing," the trace information becomes editable. If the request information is "Change," the trace information is changed and change information indicating the change is added to the changed location. If the request information is "Confirm Editing," the edited trace information is overwritten in the recording unit 118. If the request information is "Reset," the change information attached to the trace information is reset. The trace information and the attachment of change information used in this embodiment will be explained in detail with reference to Figures 5 and 6. 【0044】 Figure 5 shows an example of trace information used in this embodiment. 500 represents the number of the trace information being recorded. 501 represents the time elapsed since the start of the trace. 502 describes one of the operation information for pan, tilt, or zoom. 503 describes one of the operation information for start or stop. 504 describes one of the information for right, left, up, or down. 505 describes speed information, such as speed=500. 506 describes the target arrival position. 507 describes whether or not there has been a change. 【0045】 In this embodiment, trace information is recorded line by line, linked to time information. In this example, the pan is started to move to the right at a speed of 500 at No. 1 and No. 2, the tilt is started to move upward at a speed of 500, the tilt is stopped at No. 3, and the pan is stopped at No. 4. It also shows an example where the pan is started to move to the left at maximum speed at No. 5, and the pan is stopped at No. 6. 【0046】 The trace information will be described in a configuration suited to this embodiment, but the configuration is not limited to this one. It may also include roll information representing rotation and image quality settings such as exposure and white balance in the operation information of 502. 【0047】 Figure 6 shows an example of modifying the trace information used in this embodiment of Figure 5. 600 represents the number of the trace information being recorded. 601 represents the time elapsed since the start of the trace. 602 describes one of the operation information (pan, tilt, or zoom). 603 describes one of the operation information (start or stop). 604 describes one of the operation direction information (right, left, up, or down). 605 describes the operation speed information, such as speed=500. 606 describes the target arrival position. 607 describes start and stop as information for the start and stop of the modified section. 【0048】 This example shows an added control that starts the tilt operation at maximum speed in No. 4 and stops the tilt operation in No. 7. The modification of trace information is explained in the example of adding control operations, but if the direction information or speed information of the pan operation performed in No. 6 is changed, or if the target arrival position is changed, the change interval can also be set to the period between the target position in No. 3 (60°, 40°) and the target position in No. 8 (50°, 0°), or the change interval can be set to 5 to 20 seconds of time information. 【0049】 In this embodiment, the change information described in 607 is recorded as start and stop, but any configuration that shows the change interval is acceptable. For example, a configuration that represents the changed interval with flag information such as "0" or "1", a configuration that records start1 / stop1, start2 / stop2, etc. for each of the first and second edits, or a configuration that records the changed information (pan / tilt operation information, time information, speed information). 【0050】 The trace editing unit 403 overwrites the modified trace information in the recording unit 118. In this embodiment, the modified information is defined as information until a "reset" request is input, but the method of resetting the modified information is not limited to this. For example, it may be configured to automatically reset after playback by the trace playback unit 404 described later, or to automatically reset when editing is interrupted by the start of another function. 【0051】 As described above, the trace editing unit 403 can add combinations of trace information 602-607 and time information 601 for No. 4 and No. 7. The trace information includes target position information 606 of the imaging unit 111. The time interval modified by the trace editing unit 403 is the time interval between the target position information 606 before and after the trace information was modified. 【0052】 Trace editing unit 403 may add change information 607, a flag indicating whether or not the trace information has been modified by trace editing unit 403. Furthermore, trace editing unit 403 may assign different change information 607 to multiple trace information items that have been modified by trace editing unit 403. 【0053】 The trace information includes the type of pan or tilt 602, the direction of the pan or tilt 604, and the speed of the pan or tilt 605. The trace editing unit 403 resets the change information 607 under the predetermined conditions described above. 【0054】 The trace playback unit 404 reads the trace information recorded in the recording unit 118 from the request information input from the trace control request reception unit 401 and executes the trace playback process. If the request information is "start playback", it outputs time information and control information to the control request unit 405, and time information and change information to the change section notification unit 406. If the request information is "stop playback", it stops the output information and cancels playback. 【0055】 The control request unit 405 transmits a control request to the pan / tilt control unit 117 and the zoom control unit 114 based on the time information and control information input from the trace playback unit 404. 【0056】 The change section notification unit 406 notifies the image processing unit 112 of the time information to which the trace change information has been added, based on the time information input from the trace playback unit 404 and the change information, and the image processing unit 112 superimposes an emphasis frame on the image data. The change section notification unit 406 acquires the image data with the emphasis frame superimposed and transmits it to the communication unit 120 and the video IF control unit 121. In this embodiment, the method of notifying a change section is described as an example of notifying a change section by superimposing an emphasis frame on the image data, but it is not limited to this, and other configurations such as notification by an LED or tally lamp attached to the imaging device 100, or notification by superimposing a string of characters on the video, are also acceptable. 【0057】 Here, the procedure of the trace control unit 400 will be explained with reference to the flowchart in Figure 7. The imaging device 100 is an example of a trace control device. The control method of the imaging device 100 will be explained below. 【0058】 In step S700, the trace control request receiving unit 401 receives a trace control request transmitted from the client device 200. There are three types of control requests: "record request," "modification request," and "playback request." If the control request is a "record request" and the request information is "start recording" or "stop recording" (YES in S700), the trace control request receiving unit 401 outputs the received trace recording start request to the trace recording unit 402 and proceeds to step S701. For any other request (NO in S700), the unit proceeds to step S702. 【0059】 In step S701, the trace recording unit 402 records trace information to the recording unit 118 based on the request information input from the trace control request receiving unit 401. The trace recording unit 402 starts recording if the request information is "start recording" and stops recording if the request information is "stop recording". Then, the process proceeds to step S708. 【0060】 If the trace control request receiving unit 401 receives a request in step S700 to record trace information for the control of the imaging unit 111, the process proceeds to step S701. In step S701, the trace recording unit 402 records the trace information for the control of the imaging unit 111 in association with time information. For example, the control of the imaging unit 111 may include control of pan, tilt, zoom, or focus of the imaging unit 111. Alternatively, the control of the imaging unit 111 may include control of exposure or white balance of the imaging unit 111. 【0061】 In step S702, the trace control request receiving unit 401 receives a trace control request transmitted from the client device 200. There are three types of control requests: "record request," "modification request," and "playback request." If the control request is a "modification request" and the request information is "edit start," "edit confirmation," "modification," or "reset" (YES in S702), the trace control request receiving unit 401 outputs the request information to the trace editing unit 403 and proceeds to step S703. For any other request (NO in S702), the process proceeds to step S704. 【0062】 In step S703, the trace editing unit 403 reads the trace information recorded in the recording unit 118 from the request information input from the trace control request reception unit 401 and edits the trace information. If the request information is "Start Editing," the trace information becomes editable. If the request information is "Change," the trace information is changed, and change information indicating the change is added to the changed location. If the request information is "Confirm Editing," the edited trace information is overwritten in the recording unit 118. If the request information is "Reset," the change information attached to the trace information is reset. Then, the process proceeds to step S708. 【0063】 In step S702, if the trace control request receiving unit 401 receives a request to change the trace information for the control of the imaging unit 111, the process proceeds to step S703. In step S703, the trace editing unit 403 functions as a modification unit and modifies the trace information or time information recorded in step S701. 【0064】 In step S704, the trace control request receiving unit 401 receives a trace control request transmitted from the client device 200. There are three types of control requests: "record request," "modification request," and "playback request." If the control request is a "playback request" and the request information is "playback start" or "playback stop" (YES in S704), the trace control request receiving unit 401 outputs the request information to the trace playback unit 404 and proceeds to step S705. For any other request (NO in S704), the unit proceeds to step S700. 【0065】 In step S705, the trace playback unit 404 reads the trace information recorded in the recording unit 118 from the request information input from the trace control request reception unit 401 and executes the trace playback process. If the request information is "start playback", the trace playback unit 404 outputs time information and control information to the control request unit 405, and time information and change information to the change section notification unit 406. If the request information is "stop playback", the unit stops outputting information and cancels playback. Then, the process proceeds to step S706. 【0066】 In step S704, if the trace control request receiving unit 401 receives a request to reproduce the trace information of the imaging unit 111's control, the unit proceeds to step S705. In step S705, the trace reproduction unit 404 controls the unit to reproduce the trace of the imaging unit 111's control based on the trace information and time information recorded or modified in step S701 or S703. 【0067】 In step S706, the control request unit 405 sends a control request to the pan / tilt control unit 117 and the zoom control unit 114 based on the time information and control information input from the trace playback unit 404. Then, the process proceeds to step S707. 【0068】 The pan / tilt control unit 117 and the zoom control unit 114 control the imaging unit 111 based on the playback control in step S705. 【0069】 In step S707, the change section notification unit 406 notifies the image processing unit 112 of the time information to which the trace change information has been added, based on the time information input from the trace playback unit 404 and the change information. The image processing unit 112 then superimposes an emphasis frame onto the image data. The change section notification unit 406 acquires the image data with the emphasis frame superimposed and transmits it to the communication unit 120 and the video IF control unit 121. Then, the process proceeds to step S708. 【0070】 If the modified section notification unit 406 receives a request to modify trace information in step S702 before controlling playback in step S705, it notifies the user of information indicating the modified time section in step S703 during playback control in step S705. For example, an emphasis frame is superimposed only on the frames of image data within the modified time section. The user can recognize that the section of the frame on which the emphasis frame is superimposed is the modified time section. 【0071】 The change section notification unit 406 notifies the system by superimposing an emphasis frame onto the image data frame corresponding to the time section changed in step S703, from among the image data captured by the imaging unit 111. The change section notification unit 406 may also superimpose an emphasis frame of a different color for each control item changed in step S703. For example, control items include pan, tilt, and zoom. The change section notification unit 406 may also notify the system by superimposing a string of characters onto the image data frame corresponding to the time section changed in step S703, from among the image data captured by the imaging unit 111. The change section notification unit 406 may also notify the system using an LED or a tally lamp. 【0072】 In step S708, the trace control unit 400 determines whether to continue or stop the control function of the trace control unit 400 based on user operation. If the control function is to be continued (NO in S708), the process proceeds to step S700; if the control function is to be stopped (YES in S708), the trace control unit 400 stops its control function. 【0073】 As described above, by adding change information to the modified sections in response to a trace modification request, and by superimposing a highlighting frame on the video based on the change information when a trace playback request is received, the modified sections can be notified on the video. 【0074】 (Second Embodiment) The configuration of a second embodiment of the present invention will be described with reference to Figure 8. Figure 8 is a block diagram showing an example of the functional configuration of the trace control unit 800 included in the system control unit 203 within the client device 200 according to this embodiment. 【0075】 The trace control unit 800 generates camera control commands in response to GUI operations by the user and controls the tracing of the imaging device 100. The trace control unit 800 includes a trace control request receiving unit 801, a trace recording unit 802, a trace editing unit 803, a trace playback unit 804, a control request unit 805, a change section notification unit 806, and a change section display unit 807. 【0076】 The trace control request receiving unit 801 receives trace control requests transmitted by the user operating the GUI displayed on the display unit 201 via the input unit 202. Three types of control requests are accepted: "recording requests," "modification requests," and "playback requests." Each "recording request" has request information for "start recording" and "stop recording," each "modification request" has request information for "start editing," "confirm editing," "change," and "reset," and each "playback request" has request information for "start playback" and "stop playback." The GUI displayed on the display unit 201 used in this embodiment will be explained in detail with reference to Figure 9. 【0077】 Figure 9 shows an example of the GUI displayed on the display unit 201 used in this embodiment. 900 indicates the video display area for displaying acquired video. 901 indicates the trace recording start button. 902 indicates the trace recording stop button. 903 indicates the trace editing button. 904 indicates the trace editing content confirmation button. 905 indicates the trace editing reset button. 906 indicates the trace playback start button. 907 indicates the trace playback stop button. 908 indicates the graph display area for trace information. 909 indicates the control item tab. 910 indicates the position information axis (in this figure, it indicates the pan position). 911 indicates the time information axis. 912, 913, 914, 915, and 916 indicate the target positions that the user has specified for the trace to pass through. 917, 918, 919, and 920 indicate the trajectory information between the target positions that the trace passes through. 【0078】 Buttons 901, 902, 903, 904, 905, 906, and 907, when pressed, send request information for "Start Recording," "Stop Recording," "Start Editing," "Confirm Editing," "Reset," "Start Playback," and "Stop Playback," respectively. By pressing the edit button 903, selecting the camera control tab to be edited on 909, and manipulating the graph displayed on 908, a "Change" request is sent, and the trace information can be edited. In this embodiment, a tab-switching GUI is described, but any configuration that allows for editing of trace information is acceptable. 【0079】 If the trace control request is a "recording request," the trace control request receiving unit 801 outputs the request information to the trace recording unit 802. If the trace control request is a "modification request," the trace control request receiving unit 801 outputs the request information to the trace editing unit 803. If the trace control request is a "playback request," the trace control request receiving unit 801 outputs the request information to the trace playback unit 804. 【0080】 The trace recording unit 802 starts and stops writing trace information to the recording unit 205 based on the request information input from the trace control request reception unit 801. The trace recording unit 802 starts recording if the request information is "start recording" and stops recording if the request information is "stop recording". 【0081】 The trace editing unit 803 receives request information from the trace control request receiving unit 801, reads the trace information recorded in the recording unit 205, and edits the trace information. If the request information is "Start Editing," the trace information becomes editable. If the request information is "Change," the trace information is changed, and change information indicating the change is added to the location of the change. If the request information is "Confirm Editing," the edited trace information is overwritten in the recording unit 205. If the request information is "Reset," the change information added to the trace information is reset. An example of changing the trace information and adding change information indicating the change to the location of the change in this embodiment will be explained in detail with reference to Figures 10, 11, and 12. 【0082】 Figure 10 shows an example where change information is added when the user changes the pan position. 1000 indicates the graph display area for trace information. 1001 indicates the control item tab. 1002 indicates the position information axis (in this figure, it shows the pan position). 1003 indicates the time information axis. 1004, 1005, 1006, 1007, and 1008 indicate the trace target positions specified by the user. 1009 indicates the trace target position after the user changed it from the trace target position in 1005. 1010, 1011, 1012, and 1013 show the trajectory information between the trace target positions. 1014 and 1015 show the trajectory information between the trace target positions after changing the position information in 1005 from the trajectory information between the trace target positions in 1010 and 1011. 【0083】 If the pan position information is changed from 1005 to 1009, the trajectory will change from 1010 and 1011 to 1014 and 1015. Therefore, the change information will be added between the passing target position 1004 and the passing target position 1006. 【0084】 Figure 11 shows an example where the user changes the time information for reaching the pan position and adds the change information. 1100 indicates the graph display area for trace information. 1101 indicates the control item tab. 1102 indicates the position information axis (in this figure, it shows the pan position). 1103 indicates the time information axis. 1104, 1105, 1106, 1107, and 1108 indicate the trace target positions specified by the user. 1109 indicates the trace target positions after the user changed them from the trace target positions in 1105. 1110, 1111, 1112, and 1113 indicate the trajectory information between the trace target positions. 1114 and 1115 indicate the trajectory information between the trace target positions after changing the time information in 1105 from the trajectory information between the trace target positions in 1110 and 1111. 【0085】 If the pan position information is changed from 1105 to 1109, the trajectory will change from 1110 and 1111 to 1114 and 1115. Therefore, the change information will be added between the passing target position 1104 and the passing target position 1106. 【0086】 Figure 12 shows an example of how the user has added information when they have changed the velocity information at a specific location. 1200 indicates the graph display area for trace information. 1201 indicates the control item tab. 1202 indicates the position information axis (in this figure, it shows the pan position). 1203 indicates the time information axis. 1204, 1205, 1206, 1207, and 1208 indicate the target locations that the user has specified for the trace to pass. 1209, 1210, 1211, and 1212 indicate the trajectory information between the target locations that the trace passes. 1213 indicates the velocity information (slope) at the target location of the trace in 1205. 1214 indicates the velocity information after it has been changed from the velocity information (slope) in 1213. 【0087】 1215 and 1216 represent the trajectory information between target positions of the trace after changing the speed information at 1205 from the trajectory information between target positions of the traces at 1109 and 1110. When the pan position information is changed from 1205 to 1209, the trajectory changes from 1210 and 1211 to 1214 and 1215, so the change information is added between target positions 1204 and 1206. 【0088】 The trace editing unit 803 overwrites the trace information with the modified information in the recording unit 205. 【0089】 The trace playback unit 804 reads the trace information recorded in the recording unit 205 from the request information input from the trace control request reception unit 801 and executes the trace playback process. If the request information is "start playback", the trace playback unit 804 outputs time information and control information to the control request unit 805, and time information and change information to the change section notification unit 806. If the request information is "stop playback", the unit stops outputting the information and cancels playback. 【0090】 Based on the time information and control information input from the trace playback unit 804, the control request unit 805 transmits a control request to the imaging device 100 via the network 300 using the communication unit 204. 【0091】 The change section notification unit 806 superimposes an emphasis frame onto the image data received by the communication unit 204 from the imaging device 100 via the network 300, based on the time information input from the trace playback unit 804 and the time information to which the change information has been added. In this embodiment, the method of notifying a change section is described as notifying a change section by superimposing an emphasis frame onto the image data, but the method of notification is not limited to this. For example, in an information processing device having a display means that displays the trace trajectory on a graph represented by a time axis and a position axis, the trace trajectory may be notified by being thickened and emphasized, or the changed control item may be color-coded and emphasized for notification. The change section notification unit 806 transmits the image data with the emphasis frame superimposed to the display unit 201. 【0092】 The change section notification unit 806 may display the trace trajectory based on the trace information on a graph represented by a time axis and a position axis, and notify by displaying the trace trajectory of the time section changed by the trace editing unit 803 in a bolder, more prominent color. Alternatively, the change section notification unit 806 may display the trace trajectory based on the trace information on a graph represented by a time axis and a position axis, and notify by displaying the trace trajectory of the time section changed by the trace editing unit 803 in a different color for each control item 909 changed by the trace editing unit 803. For example, the control items 909 include pan, tilt, and zoom. 【0093】 The change section display unit 807 displays the image data input from the change section notification unit 806 on the display unit 201. 【0094】 Here, the procedure of the trace control unit 800 will be explained with reference to the flowchart in Figure 13. The client device 200 is an example of a trace control device. The control method of the client device 200 will be explained below. 【0095】 In step S1300, the trace control request receiving unit 801 receives a trace control request sent by the user operating the GUI displayed on the display unit 201 via the input unit 202. Three types of control requests are accepted: "record request," "modification request," and "playback request." If the control request is a "record request" and the request information is "start recording" or "stop recording" (YES in S1300), the trace control request receiving unit 801 outputs the received request information to the trace recording unit 802 and proceeds to step S1301. For any other request (NO in S1300), the process proceeds to step S1302. 【0096】 In step S1301, the trace recording unit 802 starts and stops writing trace information to the recording unit 205 based on the request information input from the trace control request receiving unit 801. The trace recording unit 802 starts recording if the request information is "start recording" and stops recording if the request information is "stop recording". Then, the process proceeds to step S1302. 【0097】 In step S1302, the trace control request receiving unit 801 receives a trace control request sent by the user operating the GUI displayed on the display unit 201 via the input unit 202. Three types of control requests are accepted: "record request," "modification request," and "playback request." If the control request is a "modification request" and the request information is "start editing," "confirm editing," "change," or "reset" (YES in S1302), the trace control request receiving unit 801 outputs the request information to the trace editing unit 803 and proceeds to step S1303. For any other request (NO in S1302), the process proceeds to step S1304. 【0098】 In step S1303, the trace editing unit 803 receives request information from the trace control request receiving unit 801, reads the trace information recorded in the recording unit 205, and edits the trace information. If the request information is "Start Editing," the trace information becomes editable. If the request information is "Change," the trace information is changed, and change information indicating the change is added to the changed location. If the request information is "Confirm Editing," the edited trace information is overwritten in the recording unit 205. If the request information is "Reset," the change information attached to the trace information is reset. Then, the process proceeds to step S1309. 【0099】 In step S1304, the trace control request receiving unit 801 receives a trace control request sent by the user operating the GUI displayed on the display unit 201 via the input unit 202. Three types of control requests are accepted: "record request," "modification request," and "playback request." If the control request is a "playback request" and the request information is "playback start" or "playback stop" (YES in S1304), the trace control request receiving unit 801 outputs the request information to the trace playback unit 804 and proceeds to step S1305. For any other request (NO in S1304), the process proceeds to step S1300. 【0100】 In step S1305, the trace playback unit 804 reads the trace information recorded in the recording unit 205 from the request information input from the trace control request reception unit 801 and executes the trace playback process. If the request information is "start playback", the trace playback unit 804 outputs time information and control information to the control request unit 805, and time information and change information to the change section notification unit 806. If the request information is "stop playback", the unit stops outputting information and cancels playback. Then, the process proceeds to step S1306. 【0101】 In step S1306, the control request unit 805, based on the time information and control information input from the trace playback unit 804, sends a camera control request to the imaging device 100 via the network 300 using the communication unit 204. Then, the process proceeds to step S1307. 【0102】 In step S1307, the change section notification unit 806 superimposes an emphasis frame onto the image data received by the communication unit 204 from the imaging device 100 via the network 300, based on the time information input from the trace playback unit 804 and the time information to which the change information has been added. The change section notification unit 806 transmits the image data with the emphasis frame superimposed to the display unit 201. Then, the process proceeds to step S1308. 【0103】 In step S1308, the change section display unit 807 displays the image data input from the change section notification unit 806 on the display unit 201. Then, the process proceeds to step S1309. 【0104】 In step S1309, the trace control unit 800 determines whether to continue or stop the control function of the trace control unit 800 based on user operation. If the control function is to be continued (NO in S1309), the process proceeds to step S1300. If the control function is to be stopped (YES in S1309), the trace control unit 800 stops its control function. 【0105】 As described above, when a trace modification request is received via the GUI, the modified information is added based on the changed location and trajectory information. When a trace playback request is received, the modified section can be notified on the video by superimposing a highlighting frame based on the modified information. 【0106】 As described above, according to the first and second embodiments, the trace function, which can reproduce camera control based on operation information recorded in chronological order based on information operated by the user, can notify the user of the change in the section when the recorded trace information is modified. 【0107】 It is possible to add information indicating that corrections have been made to added / modified information within the trace information, and to notify the user of the sections in the trace playback where this modified information is added. It is also possible to notify the user of any changes to the trace information. 【0108】 The present invention is not limited to the embodiments described above, and various modifications (including organic combinations of each embodiment) are possible based on the spirit of the invention, and these are not excluded from the scope of the invention. For example, in the configuration of the first embodiment, the change section may be notified by LEDs, tally lamps, or by superimposing text on the video. For example, in the configuration of the second embodiment, the change section may be highlighted by emphasizing the trajectory line shown in the graph displayed on the GUI. 【0109】 (Other embodiments) This disclosure can also be implemented by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions. 【0110】 Furthermore, the embodiments described above are merely examples illustrating how to implement this disclosure, and they should not be interpreted as limiting the technical scope of this disclosure. In other words, this disclosure can be implemented in various ways without departing from its technical concept or its main features. 【0111】 This embodiment includes the following configuration. (Item 1) When a request is received to record control trace information of the imaging unit, a recording means records the control trace information of the imaging unit in association with time information, When a request is received to change the trace information of the control of the imaging unit, the modification means modifies the trace information or time information recorded by the recording means, When a request for playback of trace information of the control of the imaging unit is received, playback means controls the playback of the trace of the control of the imaging unit based on the recorded or modified trace information and time information. If a request to change the trace information is received before playback is controlled by the playback means, a notification means notifies the user of information indicating the time interval changed by the modification means during playback control by the playback means. A trace control device characterized by having the following features. (Item 2) The trace control device according to item 1, further comprising control means for controlling the imaging unit based on the control of the playback means. (Item 3) The trace control device according to item 1 or 2, characterized in that the control of the imaging unit includes control of the pan, tilt, zoom, or focus of the imaging unit. (Item 4) The trace control device according to any one of items 1 to 3, characterized in that the control of the imaging unit includes control of the exposure or white balance of the imaging unit. (Item 5) The trace control device according to any one of items 1 to 4, characterized in that the modification means adds a combination of the trace information and the time information. (Item 6) The trace control device according to any one of items 1 to 5, characterized in that the trace information includes target position information of the imaging unit. (Item 7) The trace control device according to item 6, characterized in that the time interval changed by the modification means is the time interval between the target position information before and after the trace information was changed. (Item 8) The trace control device according to any one of items 1 to 7, characterized in that the modification means adds modification information to a flag indicating whether or not the trace information has been modified by the modification means. (Item 9) The trace control device according to any one of items 1 to 7, characterized in that the modification means assigns mutually different modification information to a plurality of trace information modified by the modification means. (Item 10) The trace control device according to any one of items 1 to 9, characterized in that the trace information includes the type of pan or tilt, the direction of the pan or tilt, and the speed of the pan or tilt. (Item 11) The trace control device according to item 8 or 9, characterized in that the modification means resets the modification information under predetermined conditions. (Item 12) The trace control device according to any one of items 1 to 11, characterized in that the notification means superimposes an emphasis frame on the image data frame corresponding to the time interval modified by the modification means among the image data captured by the imaging unit. (Item 13) The trace control device according to item 12, characterized in that the notification means superimposes a different colored highlighting frame for each control item modified by the modification means. (Item 14) The trace control device according to item 13, characterized in that the control items include pan, tilt, and zoom. (Item 15) The trace control device according to any one of items 1 to 11, characterized in that the notification means superimposes a string of characters onto the frame of image data corresponding to the time interval modified by the modification means among the image data captured by the imaging unit. (Item 16) The trace control device according to any one of items 1 to 11, characterized in that the notification means uses an LED or a tally lamp to provide notification. (Item 17) The trace control device according to any one of items 1 to 11, characterized in that the notification means displays the trace trajectory based on the trace information on a graph represented by a time axis and a position axis, and displays and notifies the trace trajectory of the time interval changed by the modification means in a thicker, more prominent form. (Item 18) The trace control device according to any one of items 1 to 11, 17, characterized in that the notification means displays the trace trajectory based on the trace information on a graph represented by a time axis and a position axis, and notifies the user by displaying the trace trajectory of the time interval changed by the modification means in a different color for each control item changed by the modification means. (Item 19) The trace control device according to item 18, characterized in that the control items include pan, tilt, and zoom. (Item 20) When a request is received to record control trace information of the imaging unit, a recording step is performed to record the control trace information of the imaging unit in association with time information, When a request is received to change the trace information of the control of the imaging unit, a change step is performed to change the trace information or time information recorded in the recording step, When a request for playback of trace information of the control of the imaging unit is received, a playback step is performed to control the system to play back the trace of the control of the imaging unit based on the recorded or modified trace information and time information. If a request to change the trace information is received before playback is controlled by the playback step, a notification step is performed during playback control by the playback step to notify the user of information indicating the time interval changed by the change step. A control method for a trace control device, characterized by having the following features. (Item 21) A program to cause a computer to function as a trace control device as described in any one of items 1 through 19. [Explanation of symbols] 【0112】 101 Pan drive mechanism 102 Tilt drive mechanism 103 Zoom drive mechanism 111 Imaging Unit 112 Image Processing Unit 113 Lens drive unit 114 Lens control unit 115 Pan drive unit 116 Tilt drive unit 117 Pan-tilt drive unit 118 Storage section 119 Control System 120 Communications Department 121 Video IF Control Unit 200 client devices 201 Display section 202 Input Section 203 System Control Unit 204 Communications Department 205 Records Section 300 Networks 400 Trace Control Unit 401 Trace Control Request Reception Unit 402 Trace Recording Section 403 Trace Editorial Department 404 Trace Playback Unit 405 Control Request Unit 406 Section Change Notification Unit 807 Changed section display section

Claims

[Claim 1] When a request is received to record control trace information of the imaging unit, a recording means records the control trace information of the imaging unit in association with time information, When a request is received to change the trace information of the control of the imaging unit, the modification means modifies the trace information or time information recorded by the recording means, When a request for playback of trace information of the control of the imaging unit is received, playback means controls the playback of the trace of the control of the imaging unit based on the recorded or modified trace information and time information. If a request to change the trace information is received before playback is controlled by the playback means, a notification means notifies the user of information indicating the time interval changed by the modification means during playback control by the playback means. A trace control device characterized by having the following features. [Claim 2] The trace control device according to claim 1, further comprising control means for controlling the imaging unit based on the control of the playback means. [Claim 3] The trace control device according to claim 1, characterized in that the control of the imaging unit includes control of the pan, tilt, zoom, or focus of the imaging unit. [Claim 4] The trace control device according to claim 1, characterized in that the control of the imaging unit includes control of the exposure or white balance of the imaging unit. [Claim 5] The trace control device according to claim 1, wherein the modification means is characterized by adding a combination of the trace information and the time information. [Claim 6] The trace control device according to claim 1, characterized in that the trace information includes target position information of the imaging unit. [Claim 7] The trace control device according to claim 6, characterized in that the time interval changed by the modification means is the time interval between the target position information before and after the trace information was changed. [Claim 8] The trace control device according to claim 1, characterized in that the modification means adds modification information to a flag indicating whether or not the trace information has been modified by the modification means. [Claim 9] The trace control device according to claim 1, characterized in that the modification means assigns mutually different modification information to a plurality of trace information modified by the modification means. [Claim 10] The trace control device according to claim 1, characterized in that the trace information includes the type of pan or tilt, the direction of the pan or tilt, and the speed of the pan or tilt. [Claim 11] The trace control device according to claim 8, characterized in that the modification means resets the modification information under predetermined conditions. [Claim 12] The trace control device according to claim 1, characterized in that the notification means superimposes an emphasis frame on the image data frame corresponding to the time interval modified by the modification means among the image data captured by the imaging unit. [Claim 13] The trace control device according to claim 12, characterized in that the notification means superimposes a different colored highlighting frame for each control item modified by the modification means to provide notification. [Claim 14] The trace control device according to claim 13, characterized in that the control items include pan, tilt, and zoom. [Claim 15] The trace control device according to claim 1, characterized in that the notification means superimposes a string of characters onto the frame of image data corresponding to the time interval modified by the modification means among the image data captured by the imaging unit. [Claim 16] The trace control device according to claim 1, characterized in that the notification means uses an LED or a tally lamp to provide notification. [Claim 17] The trace control device according to claim 1, characterized in that the notification means displays the trace trajectory based on the trace information on a graph represented by a time axis and a position axis, and displays and notifies the trace trajectory of the time interval changed by the modification means in a thicker, more prominent manner. [Claim 18] The trace control device according to claim 1, characterized in that the notification means displays the trace trajectory based on the trace information on a graph represented by a time axis and a position axis, and notifies the user by displaying the trace trajectory of the time interval changed by the modification means in a different color for each control item changed by the modification means. [Claim 19] The trace control device according to claim 18, characterized in that the control items include pan, tilt, and zoom. [Claim 20] When a request is received to record control trace information of the imaging unit, a recording step is performed to record the control trace information of the imaging unit in association with time information, When a request is received to change the trace information of the control of the imaging unit, a change step is performed to change the trace information or time information recorded in the recording step, When a request for playback of trace information of the control of the imaging unit is received, a playback step is performed to control the system to play back the trace of the control of the imaging unit based on the recorded or modified trace information and time information. If a request to change the trace information is received before playback is controlled by the playback step, a notification step is performed during playback control by the playback step to notify the user of information indicating the time interval changed by the change step. A control method for a trace control device, characterized by having the following features. [Claim 21] A program for causing a computer to function as a trace control device according to any one of claims 1 to 19.

Images