How to specify a target for real-time tracking by pausing the video

Abstract

To enable a PTZ camera to accurately select a moving object to track, even in an environment where multiple objects moving at high speed are displayed when filming sports such as soccer. [Solution] For video continuously captured by a camera 12, multiple moving objects contained in each frame are highlighted so that they can be distinguished, the highlighted images are dynamically displayed at a timing similar to real time, and a predetermined buffer time of images from before real time is held in a state where they can be called up, the dynamically displayed images are paused at a specified timing, frames of the image for a predetermined number of seconds before and after the timing are selected and displayed in response to an operation, the designation of the moving objects to be displayed is accepted, and a control signal to follow the designated moving object is sent to the camera.

Description

[Technical Field]

[0001] The present invention relates to a system for tracking and photographing a specific object with a camera in an environment where multiple objects coexist. [Background technology]

[0002] In recent years, the diversification of content has led to an increase in the number of scenes requiring video shooting, but there has also been a trend toward a shortage of specialized cameramen. For example, when shooting sports footage featuring multiple athletes, there are often multiple and changing subjects that need to be photographed, such as the players and the ball, and this requires advanced filming techniques. However, training highly skilled cameramen is not easy, and there is a growing demand for automated filming technology.

[0003] In sports where players are particularly close together, such as soccer and basketball, the mainstream footage is footage that follows a specific target, such as a single player or the ball. For this reason, automated filming requires that a single target, which is essentially constantly moving, be designated in real time and that the target be tracked and filmed continuously. For this reason, various technologies for detecting and tracking objects are being considered.

[0004] An example of an object detection and identification technology is the open-source YOLO (You Only Look Once). It divides the entire image into fine grid cells and processes them all at once, and as an output, it displays the range of detected and identified objects in the image in a bounding box (Bbox) in real time.

[0005] An example of a technology for tracking an object is IOU (intersection-over-union). The value obtained by dividing the area of ​​the overlapping part (intersection) of two regions by the area of ​​the union of the two regions is called IOU, and is used as an index of the degree of overlap.

[0006] For two frames of images in which Bboxes are output by YOLO, the IOU of the Bboxes between the frames is calculated, and pairs of Bboxes that exceed a threshold are determined to be the same subject, thereby performing tracking. This allows the user to specify a single Bbox, and then continuously track Bboxes that are determined to be the same subject as that Bbox. A PTZ camera, which adjusts up / down / left / right and zoom, automatically adjusts to keep the tracked object within the camera's field of view, allowing for automatic capture of footage that continues to track the target.

[0007] However, the user still relies heavily on human intervention when specifying the target to be tracked. It is still up to the human to decide which player or ball to focus on in each scene. However, because objects move at high speed during sports, it is quite difficult to specify the target quickly and accurately. Patent Document 1 proposes a technology that divides the display screen for selecting the target and then allows the user to quickly press an input device such as a numeric keypad or directional pad to select the target. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-233542 Summary of the Invention [Problem to be solved by the invention]

[0009] However, although the technology in Patent Document 1 can simplify the operation itself, it requires designating while tracking the video in real time, which makes it difficult to make an accurate selection when multiple targets are displayed, etc. In particular, when attempting to designate the same target using multiple cameras, it is necessary to designate the target for each of the multiple cameras in succession during the short time that the target is within the field of view of the multiple cameras, which increases the time constraints and makes the selection even more difficult.

[0010] Therefore, an object of the present invention is to enable a camera to accurately select a moving object to be tracked even in an environment where multiple objects moving at high speed are displayed. [Means for solving the problem]

[0011] This invention is a video acquisition unit that continuously acquires video captured by a camera; a highlighting processing unit that highlights a plurality of moving objects included in each frame of the acquired video in a distinguishable manner; an image display function unit that dynamically displays the highlighted image in quasi-real-time; a video storage unit that stores video for a predetermined buffer time prior to the real time in a callable state; a pause function unit that pauses the dynamically displayed video at a specified timing; a target designation function unit that selects and displays frames of the video for a predetermined number of seconds before, after, or both of the timings in response to an operation, and accepts designation of an action target to be displayed; a camera control unit that transmits a control signal to the camera to follow the designated motion target; The above problem has been solved by the first solving means, which is an operation target designation device having the above.

[0012] In addition, in the first solution, A second solution can be adopted in which, after the target designation function unit receives the designation of the motion target, a retroactive tracking calculation is performed to track the designated motion target from the frame in which the motion target is designated to the frame at the most recent time, the location of the designated motion target in the frame at the most recent time is determined, and the camera control unit transmits this to the camera.

[0013] In addition, in the first solution, performing a continuous tracking calculation for sequentially tracking all moving objects highlighted by the highlighting processing unit in all frames; When the target designation function unit receives the designation of the motion target, a third solution can be adopted in which the location of the motion target tracked up to the frame at the most recent time is determined and transmitted to the camera as the camera control unit.

[0014] Furthermore, the present invention can employ a fourth solution in which a program is provided to cause a computer to operate as the operation target designation device according to the first to third solution.

[0015] Furthermore, the present invention can employ a fifth solution, which is a system having the camera and an action object designation device that is one of the first to third solution.

[0016] Furthermore, the present invention provides: continuously acquiring video captured by a camera; a step of highlighting a plurality of moving objects included in each frame of the acquired video in a distinguishable manner; dynamically displaying the highlighted image in near real time; a step of continuously holding video for a predetermined buffer time prior to the real time in a callable state; a step of pausing the dynamically displayed video at a specified timing; a step of selectively displaying a frame of the video for a predetermined number of seconds before or after the timing, or both of the frames, in response to an operation, and accepting a designation of an action target to be displayed; transmitting a control signal to the camera to follow the designated motion target; A sixth solution can be adopted in which the above is used as a method for specifying an operation target to be executed. [Effects of the Invention]

[0017] According to the motion object designation device, the program for operating the same, and the motion object designation method of the present invention, rather than designating a motion object while tracking it in real time, it is possible to pause the video and look back and forth for a few seconds to select a frame at a timing that makes it easy to select the motion object, allowing the operator to quickly and accurately select a motion object even in video of a large number of players competing in a match, such as soccer or basketball. This is largely due to the fact that the video to be selected is not displayed as the latest frame in real time, but is displayed with a buffer time of a few seconds, allowing the operator to pause the video and look back and forth in time to select a frame at which it is easy to designate the motion object.

[0018] Existing methods can be used to track the action target specified in a specified frame up to the latest frame. If the method tracks only the specified action target, there will be a slight time lag until the camera can actually automatically track it, but the computational load on the action target designation device itself can be reduced. On the other hand, if the method continues to track all action targets displayed in all frames in sequence, the time lag can be reduced because, once a action target is specified, the camera can be quickly directed to the current position of the action target based on the tracking record of that action target. However, continuing to track all action targets in sequence increases the computational load on the action target designation device itself. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic diagram showing an embodiment of a system including an action target designation device according to the present invention; [Figure 2] A diagram showing an example of the environment in which this invention is used, including a soccer field and the layout of the installed cameras. [Figure 3] FIG. 1 is a functional block diagram of an operation target designation device according to the present invention; [Figure 4] (a) An example of a frame containing a highlight displayed on a display device. (b) An example of a frame with an overlapping moving object several seconds after (a). [Figure 5] FIG. 1 is a conceptual diagram showing the behavior on the time axis of the operation target designation method according to the present invention. [Figure 6] (a) An example of a frame at the time of pausing, (b) An illustration of selecting the previous or next frame after pausing, (c) An illustration of a camera receiving a control signal and tracking a moving object in the center of its field of view [Figure 7] An example of a situation where the display screen is divided into four parts and images from four cameras are displayed simultaneously. [Figure 8] FIG. 1 is a flowchart illustrating a program for operating the operation target designation device according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] The present invention provides a motion object designation device, a motion object designation method, and a program therefor, for easily and accurately designating a motion object that moves to a moving image captured by a camera so that the camera can automatically track and continue to capture the motion object.

[0021] FIG. 1 shows a system configuration diagram of an action object designation system having an action object designation device 11 according to a first embodiment of the present invention and cameras 12 (12a to 12d) controlled by the action object designation device 11. As shown in FIG.

[0022] The action target designation device 11 is a computer equipped with a display device 21 and an input device 22. A typical PC, tablet, smartphone, or the like equipped with a typical arithmetic unit, memory, and storage device can be used as the hardware. However, it is necessary for the device to have a wired or wireless network function in order to connect to and control the camera 12. Also, while FIG. 1 shows a single action target designation device 11, it may be configured with multiple pieces of hardware. Hardware having the same functions may be arranged in parallel, or the configuration of the action target designation device 11, which will be described later, may be shared and realized by multiple connected pieces of hardware. For example, the computational load can be distributed by sharing the processing.

[0023] The display device is preferably one that can display in color at high speed, and specific examples include a liquid crystal monitor, an organic EL monitor, etc. The display device may be housed in the same housing as the action object designation device 11 itself, or may be externally connected via a display cable.

[0024] The input device preferably has a pointing device. Although it is possible to specify an object using only a keyboard, it is preferable to have a device such as a mouse, trackball, or touch panel that can quickly specify a predetermined point in order to quickly select an object displayed on the display device.

[0025] Furthermore, the program that causes a computer to operate as the operation target designation device 11 according to the present invention is installed as a dedicated program that performs the following operations, or is executed via a web browser or the like.

[0026] The camera 12 controlled by the action target designation device 11 according to the present invention must be a camera known as a PTZ camera, which can be remotely controlled to pan horizontally (pan), tilt vertically (tilt), and zoom. There is no particular limit to the number of cameras 12, and this can be determined according to the needs of the content to be captured. This invention can also be implemented with a single camera 12. The advantage of being able to quickly designate an action target is particularly well-suited for systems with multiple cameras 12. Here, as an example, a case will be described in which cameras 12a to 12d are installed at four locations outside a soccer field as shown in FIG. 2.

[0027] The type of video content for which an action target is specified using the action target specification device 11 according to the present invention is not particularly limited. Among sports, the device is particularly suitable for use in sports involving many players, such as basketball, not just soccer. In addition, the device can also be used in live performances featuring many performers.

[0028] 3 shows a functional block diagram of an operation target designation device 11 according to the present invention connected to a camera 12. Here, cameras 12a to 12d are collectively referred to as camera 12. When multiple cameras 12 are connected, they are preferably connected in parallel.

[0029] The role of the operation target designation device 11 includes a calculation unit 31 that mainly performs calculations for image processing, a UI unit 32 related to the user interface, and a camera control unit 33 related to control of the camera 12. These functions are realized by a program that is installed or temporarily introduced into the operation target designation device 11 as hardware. The program is recorded on a tangible, non-transitory recording medium, and when executed, is read into memory and executed by the calculation device.

[0030] The action target designation device 11 has a video acquisition unit 41 that continuously acquires video captured by the camera 12. When acquiring the video, it is preferable to continue receiving the video from the camera 12 in an environment with as little time lag as possible.

[0031] The action target designation device 11 has a video storage unit 42 that stores at least a portion of the acquired video in a readable manner. The storage medium may be a volatile memory or non-volatile memory of the action target designation device 11, or a magnetic disk. However, to enable high-speed video processing, it is desirable that at least a portion of the video data be stored in an environment where it can be read and calculated at high speed, such as a non-volatile memory or solid state drive (SSD).

[0032] The motion object designation device 11 includes a highlighting processor 43 that distinguishably highlights multiple motion objects contained in each frame of a captured video. Note that a frame refers to a still image that constitutes the video. Videos are generally displayed at approximately 30 to 60 frames per second, recognizing them as moving to the user. Motion objects contained in frames essentially refer to objects that move against the background in the video, including, but not limited to, performers such as athletes and singers, non-human creatures such as horses in races and birds in nature, and balls being exchanged on fields or courts. However, this invention is best suited for use in situations where multiple motion objects appear in a video. Distinguishing a motion object means that the user can clearly visually distinguish the motion object from the background, and the motion object designation device 11 can distinguish which motion object is being selected when selecting it from the input device 22. Specific methods include surrounding the motion object with a rectangular or circular line, or changing the color of the motion object area to distinguish it from the background. For example, YOLO can be used, but is not limited to this technology. Here, we take a soccer player P as an example of the object of action, and show an example of highlighting the player by enclosing him in a rectangle as shown in Fig. 4. Fig. 4 shows a frame of a situation where part of a soccer field has been photographed by camera 12, and each of the players in the image is surrounded by a rectangle.

[0033] The video in which the action object of each frame is highlighted by the highlighting processor 43 is preferably temporarily stored in the video memory 42 and can be recalled. The action object designation device 11 has a video display function unit 46 that dynamically displays the highlighted video in quasi-real-time (the most recent time of capture). Quasi-real-time timing means that the video can be displayed with a minimal time lag due to the calculation and communication processes for the highlighting from the most recent time the video was acquired. The destination for the dynamic display is basically the display device 21, which is either stored in the same housing as the action object designation device 11 or directly connected to it, resulting in minimal time lag. During dynamic display, it is necessary to continuously store past video for a predetermined buffer time from the most recent real-time time in an area of ​​the video memory 42 that can be recalled quickly. Specifically, storing the video in non-volatile memory is preferable, as it facilitates processing by the calculation device.

[0034] The predetermined buffer time is approximately 1 to 10 seconds. Here, it is abbreviated to "several seconds." While the video display function unit 46 displays the video captured by the camera 12 in quasi-real-time, the video storage unit 42 continues to hold video frames from several seconds prior to the real-time (latest time) so that they can be retrieved and displayed in response to an instruction. Specifically, the frames are continuously buffered in an area of ​​the video storage unit 42 that can be quickly recalled. This allows the user to recall frames from the video storage unit 42 immediately after pausing the video being viewed on the display device 21, thereby returning the displayed video to a timeline of several seconds earlier (i.e., to the past). After pausing, the video up to the most recent time is also continuously buffered in an area of ​​the video storage unit 42 that can be quickly recalled. By making it possible to recall frames from the frame at the moment of pause to the most recent time, it is possible to advance (i.e., forward or in the future) beyond the paused frame.

[0035] The buffer time may be longer than 10 seconds, but going back beyond 10 seconds may result in the moving object moving too far from the paused frame, possibly even going outside the field of view, limiting the effectiveness of assisting in the specification of the moving object. Furthermore, there is a risk that too many frames may be kept in an area that can be quickly recalled, resulting in too much execution memory and a heavy operational load. For this reason, a buffer time of 10 seconds or less is preferable, and 5 seconds or less is practically suitable in most cases. On the other hand, if the buffer time is shorter than 1 second, the time span available for going back in time becomes too short, resulting in little difference from real-time display, and the benefits of the present invention are not fully realized.

[0036] A conceptual diagram of this change on the time axis is shown in Figure 5. The frames displayed on the display device 21 also progress along with the latest time of shooting by the camera 12, but continue to be later than the buffer time.

[0037] To achieve the above operation, the action target designation device 11 includes a pause function unit 47 that pauses the dynamically displayed highlighted image at a specified timing. This pause timing can be input from the input device 22. Examples of the input device 22 include clicking or touching a button within the displayed image or a button for manipulating the image using a mouse or touch panel. The pause command can also be received by pressing a button on a keyboard, numeric keypad, or other device. This is because the input must be as prompt as possible and as close to the user's desired timing as possible. If the operation takes too long, the desired timing may be off, making it difficult to select the action target. This difference is explained using a comparison of Figures 4(a) and 4(b). Assume that the player indicated by the bold line in the figure is to be selected. If the player can be stopped at the timing shown in Figure 4(a), selection within the rectangle would be easy. However, if a delay of a few seconds results in the positional relationship shown in Figure 4(b), the player will overlap with the player to the right, making it difficult to accurately select the player indicated by the bold line in the figure. Furthermore, this positional relationship may persist for some time.

[0038] To address this, the action target designation device 11 selectively displays frames from the video within a predetermined period of time before, after, or both of the pause timings designated by the user in response to an operation, and includes a target designation function unit 48 that accepts the designation of the action target to be displayed. Selectively displaying frames in response to an operation refers to switching the display to frames from the past or future in chronological order in response to input from the input device 22. Even if the frame at the pause timing has overlapping rectangles highlighting players as shown in FIG. 6(a), it is highly likely that frames rewound several seconds before (in the past) the pause timing will have frames in which the rectangles highlighting players no longer overlap, as shown in the forefront frame in FIG. 6(b). In this way, frames from several seconds before (in the past), after (in the future), or both of the pause timing are switched and displayed. A method for accepting this selection is preferably one that allows for chronological forward and backward movement based on input from the input device 22, which can easily fast-forward and rewind the time series. Operations such as moving the mouse wheel back and forth or sliding a slide bar displayed on a touch panel back and forth or left and right can be suitably used. Furthermore, "accepting the designation of a displayed action target" as described above refers to accepting the designation of an action target that is to be tracked by the camera 12 from among the action targets displayed in the frame. The designation can be accepted by selecting a highlighted action target (here, within a rectangular frame) using the input device 22, which is a pointing device. In FIG. 6(b), a finger mark that serves as a mouse cursor is displayed as an example. The action target is designated by selecting a frame that makes it easy to designate the action target to be designated (here, player P1 surrounded by a thick frame), and moving and clicking the mouse cursor within the frame of the action target in the frame.

[0039] The motion object designation device 11 has a tracking function unit 49 that performs tracking of the motion object designated in the displayed frame to the position displayed in the latest video of the camera 12. The designated frame is a frame from several seconds ago, and the position of the motion object often moves within a few seconds, so it is necessary to track the position to which the camera 12 is actually pointed to the most recent position of the designated motion object. The tracking method is preferably the IOU method as described above, but is not particularly limited.

[0040] Tracking methods include a delayed tracking calculation method that tracks only the specified moving object, and a continuous tracking calculation method that continues to track all moving objects highlighted by the highlighting processing unit 43 in sequence. The former has a small calculation load, but there is a time lag between selection and issuing an instruction to the camera 12. The latter has a large calculation load, but can issue an instruction to the camera 12 quickly after selection.

[0041] The following explains the method of the delayed tracking calculation. After the target designation function unit 48 accepts the designation of a motion target, it tracks the designated motion target from the frame in which the motion target was designated to the frame at the most recent time. That is, for the designated motion target, it checks for motion targets that are determined to be the same subject between the designated frame and the next frame, checks for motion targets that are determined to be the same subject between the next frame and the frame after that, and so on, repeating this process up to the frame at the most recent time to determine the location (coordinates) of the designated motion target in the frame at the most recent time.

[0042] Next, the above-mentioned continuous tracking calculation method will be explained. Without waiting for the target designation by the target designation function unit 48, the relationship between all target objects highlighted by the highlighting processing unit 43 and those determined to be the same subject between frames is continuously confirmed. In other words, the movement of all target objects is continuously tracked between all frames. This means that once a target object is designated in any past frame, its location in the most recent frame can be immediately confirmed unless it has moved out of the field of view at some point. For example, it is preferable that each target object be identifiable by an ID or the like. Then, when the target designation function unit 48 accepts the designation of a target object, the location (coordinates) of that target object in the most recent frame is determined, as described above, for that target object that has already been tracked up to the most recent frame.

[0043] In either of the above methods, the location of the specified motion object in the most recent frame is determined. Motion object designation device 11 has a camera control unit 33 that transmits a control signal to camera 12 to set the coordinates of the location at an appropriate position within the angle of view of camera 12. This camera control unit 33 then continues to transmit a control signal to camera 12 to track the specified motion object. Specifically, this camera control unit 33 has a control signal calculation unit 50 that calculates the above-mentioned control signal, and a control signal transmission unit 51 that transmits the control signal to camera 12.

[0044] After receiving the control signal, camera 12 automatically operates so that the target object is displayed in the center of the image (FIG. 6(c)). This may be done automatically by camera 12, or camera control unit 33 may operate by cooperating with tracking function unit 49 based on information from highlighting processing unit 43 to generate and transmit a control signal in real time each time.

[0045] The motion target designation device 11 of the present invention allows for accurate and high-speed designation of motion targets by pausing and moving forward or backward to a frame where it is easier to designate the motion target, enabling a single user (operator) to accurately designate motion targets for multiple cameras 12. For example, as shown in Figure 7, the images from multiple cameras 12 (12a-12d) are displayed in separate views on the screen, and immediately after a scene change, motion targets can be designated one after another for the images from the multiple cameras 12 and control signals can be sent to the cameras 12. Thereafter, motion targets can be automatically tracked without user intervention until a major change in the screen occurs. According to this invention, even if a user is not particularly skilled, it takes approximately 2 to 5 seconds to designate a motion target for one camera 12.

[0046] An example of the flow of a program for operating a system consisting of an action target designation device 11 and a camera 12 will be described with reference to Fig. 8. First, the video acquisition unit 41 continuously acquires video captured by the camera 12 (S11). Receiving push-type network transmission from the camera 12 reduces the load on the action target designation device 11. At this time, it is preferable that identification information, primarily for identifying the camera 12, is also received along with the video data of each frame that constitutes the video. For this acquired video, the highlighting processing unit 43 highlights multiple action targets contained in each frame so that they can be distinguished (S12).

[0047] Then, the video display function unit 46 of the action target designation device 11 continues to secure the video for the above-mentioned predetermined buffer time (several seconds) (S13), and transmits the highlighted video to the display device 21 at a timing similar to real time, and dynamically displays it (S14). The user thereby observes the video in which the action target is highlighted, slightly delayed from real time. Here, for convenience, S13 is listed first, but in reality, S13 and S14 are executed almost in parallel. The pause function unit 47 of the action target designation device 11 pauses the video dynamically displayed on the display device 21 at the timing instructed by the user operating the input device 22 (S15).

[0048] After the pause, the target designation function unit 48 of the action target designation device 11 transmits to the display device 21 and displays the frames selected by the user's operation for a predetermined number of seconds before the pause, in accordance with the user's operation to move forward or backward on the time axis from the input device 22 (S16). Also, in (S16), if the frames after the pause up to the most recent time within the range of time that has elapsed since the pause are selected by operation, they are similarly transmitted to the display device 21 and displayed. From among the displayed frames, the designation of the action target by the input device 22 is accepted (S17).

[0049] Here, a method of delayed tracking calculation will be described as an example. Tracking function unit 49 of motion object designation device 11 performs delayed tracking calculation to track the designated motion object from the frame in which the motion object is designated to the most recent frame, and calculates and determines the position coordinates of the designated motion object in the most recent frame (S21). Camera control unit 33 of motion object designation device 11 uses these position coordinates to calculate a control signal to camera 12, which is a PTZ camera, so that the position coordinates of the designated motion object are located at the center of the video captured by camera 12 (S22). This calculated control signal is transmitted as a signal to camera 12 (S23).

[0050] Thereafter, the camera 12 or the camera control unit 33 continues to control the position coordinates of the designated motion target so that the position coordinates of the designated motion target continue to be located at the center of the video captured by the camera 12. [Explanation of symbols]

[0051] 11 Operation target designation device 12, 12a, 12b, 12c, 12d Camera 21 Display device 22 Input Devices 31 Arithmetic section 32 UI section 33 Camera control unit 41 Video acquisition unit 42 Video storage unit 43 Highlighting processing section 46 Video display function section 47 Pause function unit 48 Target Designation Function Unit 49 Tracking function unit 50 Control signal calculation unit 51 Control signal transmitter

Claims

1. a video acquisition unit that continuously acquires video captured by a camera; a highlighting processing unit that highlights a plurality of moving objects included in each frame of the acquired video in a distinguishable manner; an image display function unit that dynamically displays the highlighted image in quasi-real-time; a video storage unit that stores video for a predetermined buffer time prior to the real time in a callable state; a pause function unit that pauses the dynamically displayed video at a specified timing; a target designation function unit that selects and displays frames of the video for a predetermined number of seconds before, after, or both of the timings in response to an operation, and accepts designation of an action target to be displayed; a camera control unit that transmits a control signal to the camera to follow the designated motion target; An operation target designation device having the same.

2. 2. The motion object designation device according to claim 1, wherein after the object designation function unit receives the designation of the motion object, it performs a retroactive tracking calculation to track the designated motion object from the frame in which the motion object is designated to the frame at the latest time, determines the position of the designated motion object in the frame at the latest time, and transmits the determined position to the camera as the camera control unit.

3. performing a continuous tracking calculation for sequentially tracking all moving objects highlighted by the highlighting processing unit in all frames; 2. The motion target designation device according to claim 1, wherein when the target designation function unit receives the designation of the motion target, the motion target location tracked up to the most recent frame is determined and transmitted to the camera as the camera control unit.

4. A program for causing a computer to operate as the operation target designation device according to any one of claims 1 to 3.

5. A system comprising the camera and the action object designation device according to any one of claims 1 to 3.

6. continuously acquiring video captured by a camera; a step of highlighting a plurality of moving objects included in each frame of the acquired video in a distinguishable manner; dynamically displaying the highlighted image in near real time; a step of continuously holding video for a predetermined buffer time prior to the real time in a callable state; a step of pausing the dynamically displayed video at a specified timing; a step of selectively displaying a frame of the video for a predetermined number of seconds before or after the timing, or both of the frames, in response to an operation, and accepting a designation of an action target to be displayed; transmitting a control signal to the camera to follow the designated motion target; How to specify the target of the action to be performed.

Images