Control device, operation method for control device, and program

Abstract

A control device according to the present invention comprises: a detection means for detecting a change in, from among subjects respectively captured by a plurality of imaging devices, at least one subject to be captured by the imaging devices; and a control means for, when the change in the subject to be captured has been detected by the detection means, issuing a notification indicating, on a display screen that displays a video captured by each of the plurality of imaging devices, a video from the imaging device in which the change in the subject to be captured has occurred.

Description

Control device, operation method of control device, and program

[0001] The present disclosure relates to a control device, an operation method of the control device, and a program.

[0002] Conventionally, in shooting in a studio or the like, there is a multi-camera shooting system that automatically shoots while tracking a plurality of subjects with a plurality of cameras. The videos shot by the plurality of cameras are connected to a video confirmation monitor, and shooting is performed while displaying the plurality of videos on the video confirmation monitor to confirm that there is no abnormality in the shot videos.

[0003] In such a multi-camera shooting system, when the positions of a plurality of subjects are swapped due to the movement of the subjects, the cameras that track them may also be swapped accordingly. When the cameras are swapped, the positions where the subjects are displayed on the video confirmation monitor are swapped, so there is a case where the shooting staff who checks the videos may lose sight of the subjects they are paying attention to.

[0004] In Patent Document 1, it is disclosed that in order to prevent a monitor from losing sight of a subject tracked by a surveillance camera system, the videos of the cameras at the movement destination of the subject are rearranged and displayed in ascending order of distance.

[0005] Japanese Patent Application Laid-Open No. 2011-145730

[0006] However, in the technique described in Patent Document 1, there is a problem that it is difficult to prevent the loss of sight of a subject when the positions of a plurality of subjects are swapped due to the movement of the subjects during shooting while tracking the plurality of subjects with a plurality of cameras.

[0007] The present disclosure has been made in view of the above problems, and provides a technique for suppressing the loss of sight of a subject.

[0008] A control device according to one aspect of the present disclosure that achieves the above objectives is characterized by comprising: detection means for detecting that the subject being photographed by at least one of the subjects being photographed by a plurality of photographing devices has changed; and when the detection means detects a change in the subject being photographed, control means for notifying the display screen that displays the images taken by each of the plurality of photographing devices to show the image of the photographing device in which the change in the subject being photographed occurred.

[0009] Other features and advantages of the technical ideas derived from this disclosure will become apparent from the following description with reference to the attached drawings. In the attached drawings, the same or similar components are given the same reference numeral.

[0010] The attached drawings are included in the specification and constitute a part thereof, illustrating embodiments in this disclosure and used to explain the technical ideas derived from this disclosure together with their descriptions. A diagram showing an example of the overall configuration of a shooting system according to one embodiment. A diagram illustrating an example of the camera arrangement of the shooting system according to Embodiment 1. A diagram illustrating an example of the operation of the shooting system according to Embodiment 1. A diagram illustrating an example of the monitor display screen of the shooting system according to Embodiment 1. A flowchart showing an example of the processing flow of the shooting system according to Embodiment 1. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 1. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 1. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 1. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 1. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 1. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 2. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 2. A diagram illustrating an example of the notification display method of the shooting system according to Embodiment 3. A diagram illustrating an example of the hardware configuration of a control device according to one embodiment.

[0011] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the scope of the claims. While the embodiments describe multiple features, not all of these features are necessary, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0012] (Embodiment 1) <Overall configuration of the multi-camera shooting system and functional configuration of the control device> The multi-camera shooting system according to this embodiment will be described below with reference to the figures.

[0013] Figure 1 shows an example of the overall configuration of the multi-camera shooting system according to this embodiment. The multi-camera shooting system includes a control device 101, a LAN 105, an overhead camera (overhead shooting device) 106, a tracking camera (shooting device) A107, a tracking camera (shooting device) B108, a tracking camera (shooting device) C109, a video switcher 110, and a monitor 111.

[0014] The control device 101 controls the entire system. The control device 101 includes a tracking target determination unit 102, a tracking target change detection unit 103, and a notification display control unit 104. Details of the operation of the tracking target determination unit 102, the tracking target change detection unit 103, and the notification display control unit 104 will be described later. The control device 101 is connected to the overhead camera 106, tracking camera A 107, tracking camera B 108, tracking camera C 109, video switcher 110, monitor 111, etc. via LAN 105 (Local Area Network).

[0015] The overhead camera 106 is a camera that captures an overhead view of the entire shooting area. The captured video data is output via the video signal interface and / or LAN 105. The video signal interface is SDI (Serial Digital Interface), HDMI (High-Definition Multimedia Interface) (registered trademark), etc.

[0016] Tracking cameras A107, B108, and C109 are cameras with automatic tracking capabilities that automatically track and photograph subjects instructed by the control device 101. The tracking cameras may include PTZ cameras capable of pan, tilt, and zoom (PTZ) control. Note that the tracking cameras do not necessarily need to have automatic tracking capabilities; the control device 101 may control the PTZ cameras to achieve automatic tracking. Furthermore, while this embodiment shows an example with three tracking cameras, the number is not limited.

[0017] The tracking target determination unit 102 of the control device 101 detects subjects such as people from the overhead image received from the overhead camera 106. Then, based on the position of the subject and information such as the distance to tracking cameras A107, B108, and C109, it determines which subject each tracking camera will photograph. It then instructs each tracking camera to photograph which subject. One method for determining which subject each tracking camera will photograph is to select subjects such that the sum of the distances between each tracking camera and the subject to be photographed is minimized. In this way, all subjects can be photographed by tracking cameras located relatively close to each other.

[0018] If there are more subjects than the number of tracking cameras, the control device 101 may limit the subjects to be photographed according to a pre-set priority based on the characteristics of the subjects and the scene, or it may control the PTZ so that multiple subjects can be photographed with one tracking camera. On the other hand, if there are fewer subjects than the number of tracking cameras, multiple cameras may photograph the same subject.

[0019] The video switcher 110 receives video data captured from the overhead camera 106, tracking camera A 107, tracking camera B 108, and tracking camera C 109 via the video signal interface and LAN 105. It then arranges these four video data into a single video data and outputs it.

[0020] The monitor 111 receives video data output from the video switcher 110 and displays it on the screen. The person reviewing the recorded footage checks the image displayed on the monitor 111 to confirm that there are no abnormalities in the recording.

[0021] In this embodiment, an example with one monitor 111 is shown, but there may be multiple monitors 111. In that case, the video switcher 110 may output the input video data by distributing it into multiple video data.

[0022] Here, Figure 2 illustrates an example of camera setup for the shooting system according to Embodiment 1. It shows an example of shooting three subjects on a stage (subject A, subject B, and subject C, respectively). The overhead camera 106 shoots the entire stage, while the tracking cameras A 107, B 108, and C 109 each shoot subjects that are close to them (subject A, subject B, and subject C, respectively).

[0023] The tracking target change detection unit 103 of the control device 101 detects changes in the subject being tracked by each tracking camera and outputs information about the tracking camera and subject that have changed. The method for detecting changes in the subject being tracked may be, for example, receiving instruction information for the subject to be tracked from the tracking target determination unit 102 and detecting the change, or analyzing images taken by each tracking camera to detect changes in the subject being tracked.

[0024] The notification display control unit 104 of the control device 101 issues instructions to provide an auxiliary display to ensure that the viewer of the monitor 111 does not miss the subject they are focusing on, when the tracking target change detection unit 103 detects a change in the subject being tracked. Specifically, for example, if the subject being tracked by the tracking camera B 108 changes from subject B to subject C, the notification display control unit 104 instructs the tracking camera B 108 to output the video data with an outline for a certain period of time. The tracking camera B 108 complies with this instruction and outputs the video data with an outline for a certain period of time. On the monitor 111 screen, the video data from the tracking camera B 108 is displayed with an outline, so the viewer of the monitor 111 is alerted. To enhance the effect of alerting the viewer, the outline may be made to blink. The aforementioned certain period of time is, for example, about 1 to 2 seconds, but it may be any other length of time that is effective in alerting the viewer.

[0025] Figure 3 is a diagram illustrating an example of the operation of the shooting system according to Embodiment 1. Here, we will explain a series of operations when the order of positions of subjects A, B, and C changes from the state shown in Figure 2, where subjects A, B, and C are arranged from left to right, to the state shown in Figure 3, where the order of positions of subjects B and C is swapped. When the positions of subjects B and C are swapped, the distance between tracking camera B108 and subject C becomes shorter, and the distance between tracking camera C109 and subject B also becomes shorter.

[0026] Therefore, the tracking target determination unit 102 instructs tracking camera B 108 to track subject C, and tracking camera C 109 to track subject B. The tracking target change detection unit 103 detects that the subject being tracked by tracking camera B 108 and the subject being tracked by tracking camera C 109 have switched places. Accordingly, the notification display control unit 104 instructs that an outline be added to the video data from tracking camera B 108 and the video data from tracking camera C 109.

[0027] <Monitor Display Examples> Figures 4A and 4B illustrate an example of the monitor display of the shooting system according to Embodiment 1. Figure 4A shows an example of the screen display of monitor 111 before the positions of subject B and subject C are swapped. Figure 4B shows an example of the screen display of monitor 111 after the positions of subject B and subject C are swapped.

[0028] The upper left display area of ​​the screen shows the video from tracking camera A107, the upper right display area shows the video from tracking camera B108, the lower left display area shows the video from tracking camera C109, and the lower right display area shows the video from overhead camera 106. As can be seen from this diagram, if the tracking camera changes due to a change in the subject's position on the stage, the display area on the monitor 111 in which the subject is displayed will change. In this example, the display areas in which subjects B and C are displayed are swapped before and after the subject's position changes, so there is a possibility that viewers of the monitor 111 may lose sight of subjects B and C. In this embodiment, when the subject being tracked changes, an outer frame is displayed in the respective video data display areas of tracking camera B108 and tracking camera C109.

[0029] In other words, if a swap is detected, the monitor 111 displays a notification on its screen indicating the display positions of the images from tracking camera B108 and tracking camera C109 where the swap occurred. More specifically, the notification is made by displaying an outer frame around the display area of ​​the images from tracking camera B108 and tracking camera C109.

[0030] This ensures that the viewer of monitor 111 is alerted to both display areas, preventing them from losing sight of the subject.

[0031] <Hardware Configuration of Control Device> Next, with reference to Figure 9, an example of the hardware configuration of a control device according to one embodiment will be described. The control device 101 includes a CPU (Central Processing Unit) 11, ROM (Read-Only Memory) 12, RAM (Random Access Memory) 13, auxiliary storage device 14, operation unit 15, communication I / F 16, and bus 17.

[0032] The CPU 11 controls the entire control unit 101 using computer programs and data stored in the ROM 12 and RAM 13, thereby realizing each function of the control unit 101 shown in Figure 1. The control unit 101 may have one or more dedicated hardware components separate from the CPU 11, and at least a portion of the processing performed by the CPU 11 may be executed by the dedicated hardware. Examples of dedicated hardware include ASICs (Application-Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and DSPs (Digital Signal Processors). The ROM 12 stores programs that do not require modification. The RAM 13 temporarily stores programs and data supplied from the auxiliary storage device 14, as well as data supplied from external sources via the communication interface 16. The auxiliary storage device 14 is, for example, a hard disk drive and stores various types of data such as image data and audio data.

[0033] The control unit 15 consists of, for example, a keyboard, mouse, joystick, touch panel, etc., and receives various instructions from the user and inputs them to the CPU 11. The communication interface 16 is used for communication between the control device 101 and external devices. For example, if the control device 101 is connected to an external device by wire, a communication cable is connected to the communication interface 16. If the control device 101 has a function to communicate wirelessly with an external device, the communication interface 16 is equipped with an antenna. The bus 17 connects the various parts of the control device 101 and transmits information.

[0034] In this embodiment, the operation unit 15 is included in the control device 101, but the operation unit 15 may exist as a separate device outside the control device 101. In this case, the CPU 11 may operate as an operation control unit that controls the operation unit 15.

[0035] <Processing> Next, the processing of the imaging system according to this embodiment will be described with reference to the flowchart in Figure 5.

[0036] First, in step 101, the tracking target determination unit 102 analyzes the video data captured by the overhead camera 106 to determine the subject that each tracking camera A107 to C109 will photograph.

[0037] In step S 102, the tracking target change detection unit 103 determines whether the subject being photographed by each tracking camera A107 to C109 has changed. If a change in subject is detected, the process proceeds to step S 103. If no change in subject is detected, the process ends.

[0038] In step S 103, the notification display control unit 104 instructs the tracking camera, whose subject has changed, to display a notification on the captured video data.

[0039] In step 104, the tracking camera that has received instructions to display a notification outputs the video data to the monitor 111 with an outline frame for a certain period of time. If there is a change in the subject being filmed, the outline frame will be displayed in the display area of ​​the tracking camera's monitor 111, thus alerting the viewer of the monitor 111.

[0040] In the example described above, the notification display control unit 104 issued instructions to the tracking camera, and the tracking camera added an outline to the video data. However, the entity that performs the process of adding an outline to the video data is not limited to the tracking camera. For example, the notification display control unit 104 may control the video switcher 110, and the video switcher 110 may perform the control to add an outline to the video data. When the video switcher 110 adds an outline to the video data, the tracking camera can output unprocessed video data, which has the advantage of saving the video data as it was captured when recording the video data.

[0041] Furthermore, the method for notifying the viewer of the monitor 111 of a change in the subject is not limited to adding an outline to the video data. Figures 6A to 6D illustrate an example of a notification display method for a multi-camera shooting system according to Embodiment 1.

[0042] For example, as shown in Figure 6A, when a change in the subject occurs, information (a shape or symbol) indicating the display position of the new screen may be superimposed on each video data to notify the user.

[0043] In other words, information (a shape or symbol) indicating the position of the display area of ​​the image of tracking camera C109 on which subject B will be displayed after the switch may be displayed in the display area of ​​the image of tracking camera B108 on which subject B was displayed before the switch. Similarly, information (a shape or symbol) indicating the position of the display area of ​​the image of tracking camera B108 on which subject C will be displayed after the switch may be displayed in the display area of ​​the image of tracking camera C109 on which subject C was displayed before the switch. In this case, the information (a shape or symbol) may be highlighted by flashing.

[0044] Also, for example, in FIG. 6B, information indicating the angle of view of each tracking camera (broken line frame) and information indicating the display position on the display screen of that angle of view (figure or symbol) in the vicinity (upper right) of each piece of information are displayed in the video data of the overhead camera 106. In such a case, when a subject replacement occurs, the information (frame) indicating the angle of view of the corresponding tracking camera may be highlighted and notified, such as by flashing the display. By observing the flashing display of the frame indicating the angle of view and the display position of the camera image in the upper right of that frame, it is possible to easily recognize which camera's image has been replaced. Also, the information (frame) indicating the angle of view may be displayed in different colors, or marked with symbols, etc., to make the correspondence between the information (frame) indicating the angle of view and the display area of the corresponding tracking camera easier to understand. Alternatively / Furthermore, the information (figure or symbol) indicating the display position on the display screen displayed in the vicinity (upper right in the illustrated example) of the information (frame) indicating the angle of view may be further flashed and displayed.

[0045] Also, for example, as shown in FIG. 6C, identification information (identification number or identification symbol, etc.) may be assigned to each subject in advance, and a map that can list which display area each subject is displayed in may be displayed. That is, the display screen of the monitor 111 may include a map that can list which camera's video display area each subject is displayed in. And when a subject replacement occurs, it may be notified by flashing the display on the map corresponding to the subject. In the example of FIG. 6C, the identification information (B, C) of subjects B and C in which a subject replacement has occurred is flashed and notified. In this way, when a replacement is detected, the notification may be made by highlighting the position on the map corresponding to each display area of the video of each camera in which the replacement has occurred. And the identification information of subject C that the tracking camera B108 is shooting after the replacement may be flashed and displayed at the first position (upper right position) on the map corresponding to the display area of the video of the tracking camera B108 in which the replacement has occurred. Furthermore, the notification may be made by flashing and displaying the identification information of subject B that the tracking camera C109 is shooting after the replacement at the second position (lower left position) on the map corresponding to the display area of the video of the tracking camera C109 in which the replacement has occurred.

[0046] Also, as shown in FIG. 6D, identifiers or the like may be assigned to each subject in advance, and a unique frame (for example, a solid line frame for subject A, a dashed line frame for subject B, a dotted line frame for subject C, etc.) may be added according to the identifier of the subject being photographed, and the video data may be displayed. Then, when the subject is replaced, the corresponding frame of the display area (for example, the dashed line frame of subject B, the dotted line frame of subject C) may be blinked and displayed to notify. That is, a unique frame for each subject may be displayed for the display area of the video of each camera on the display screen, and the frame of each display area where the replacement has occurred may be highlighted to notify.

[0047] Note that the methods shown in FIGS. 6B and 6C have the advantage that since it is not necessary to process the video data captured by each tracking camera, the original video data captured can be saved when recording the video data.

[0048] As described above, in the present embodiment, when the subject captured by the camera changes (changes to a different subject) due to the replacement of the position of the subject in multi-camera shooting, the monitor that displays the captured video is notified that the replacement has occurred. Thereby, the person who is checking the captured video can continue to check the video without losing sight of the subject.

[0049] (Embodiment 2) In Embodiment 1, an example in the case where the camera to be used for shooting is replaced due to the replacement of the position of the subject has been described. On the other hand, for example, when the number of subjects is larger than the number of tracking cameras, the subject that has been photographed may no longer be a shooting target due to the replacement of the position of the subject. In that case, the subject that is no longer a shooting target will disappear from the screen of the monitor 111, so the person who was viewing that subject on the monitor 111 will have to search for the whereabouts of the subject. In the present embodiment, a notification method in the case where the subject that has been displayed on the monitor 111 is no longer displayed due to the replacement of the position of the subject or the like will be described.

[0050] An example of the configuration of the multi-camera shooting system according to this embodiment is the same as in Embodiment 1. The operation of the tracking target determination unit 102, LAN 105, overhead camera 106, tracking camera A 107, tracking camera B 108, tracking camera C 109, video switcher 110, and monitor 111 according to this embodiment is the same as in Embodiment 1.

[0051] In this embodiment, the tracking target change detection unit 103 detects a subject that is no longer being tracked when the subject being tracked by each tracking camera changes, and outputs information of the tracking camera that was previously photographing that subject.

[0052] In this embodiment, the notification display control unit 104, when the tracking target change detection unit 103 detects that the subject that was being tracked is no longer being tracked, issues an instruction to display an auxiliary indication on the monitor 111 that the subject is no longer being photographed. This prevents the person viewing the monitor 111 from needlessly searching for the whereabouts of the subject they are interested in.

[0053] Specifically, if the subject being tracked by the tracking camera C109 changes from subject C to another subject, and there is no longer a tracking camera tracking subject C, the notification display control unit 104 instructs the tracking camera C109 to output the video data with an outline for a certain period of time. The tracking camera C109 complies with this instruction and outputs the video data with an outline for a certain period of time indicating that the subject is no longer the target of filming. On the monitor 111 screen, the video data from the tracking camera C109 is displayed with an outline to draw the attention of the person viewing the monitor 111. Instead of an outline, or in addition to an outline, symbols or characters indicating that the subject is no longer the target of filming may be added.

[0054] Figures 7A to 7D illustrate an example of a notification display method for the multi-camera shooting system according to this embodiment.

[0055] Figure 7A shows an example of the screen display on monitor 111 before the subject being tracked by the tracking camera changes. This example shows a case where there are four subjects, with subject D added compared to Figure 4A. The overhead camera 106 shows all four subjects A through D, the tracking camera A 107 shows subject A, the tracking camera B 108 shows subject B, and the tracking camera C 109 shows subject C.

[0056] Figure 7B shows an example of the display on the monitor 111 after the subject being filmed by the tracking camera C109 changes from subject C to subject D. Since subject C is no longer displayed in the image from the tracking camera C109, a frame is displayed on the image from the tracking camera C109 that was previously filming subject C. Alternatively, a message indicating that the subject that was previously being filmed is no longer being filmed may be displayed, such as a SWITCH icon. Alternatively, either one of these displays may be shown, or both may be displayed in combination.

[0057] Furthermore, for example, in Figure 7C, the image from the overhead camera 106 displays frames (dashed lines) indicating the field of view of each tracking camera, and near each piece of information (upper right) is information (a shape or symbol) indicating the display position of that field of view on the display screen. In such a case, the information (a shape or symbol) indicating the display position of the image from the tracking camera C109 that was capturing the subject C is highlighted (flashed). At the same time, the subject C that is no longer captured by any of the cameras is highlighted on the image from the overhead camera 106 (for example, the frame surrounding subject C is flashed). That is, the information indicating the position of the display area on the display screen of the image from the camera that was capturing subject C that is no longer captured by any of the cameras (in the example shown, a shape with a black square in the lower left) is highlighted. At the same time, the information indicating the position of the subject C that is no longer captured by any of the cameras on the image from the overhead camera 106 (a dashed line frame surrounding subject C) is highlighted. This makes it easy to recognize that subject C is no longer the target of tracking camera C109, and furthermore, that it is no longer being photographed by any camera.

[0058] In addition, in Figure 7C, the system may be configured to highlight only the dashed frame surrounding subject C. This makes it easy to recognize that subject C is no longer being captured by any of the cameras.

[0059] Furthermore, as shown in Figure 7D, for example, identification information (such as an identification number or identification symbol) may be assigned to each subject in advance, and a map may be displayed that shows which display area each subject is shown in. When the subject being tracked by the tracking camera changes and the subject that was previously being tracked is no longer the target of tracking, the identification information of the subject on the map display that previously showed that subject may be notified by flashing. Alternatively, a message indicating that the subject is no longer the target of shooting may be displayed. For example, in Figure 7D, since subject C, which was being photographed by tracking camera C109, has changed to subject D, the identification information of the changed subject, "D," may be flashed on the map corresponding to tracking camera C. Alternatively, the message "SWITCH" indicating that the subject is no longer the target of shooting may be displayed on the same map.

[0060] Furthermore, the methods shown in Figures 7C and 7D have the advantage of not requiring any processing of the video data captured by each tracking camera, allowing the video data to be saved in its original form when recording it.

[0061] In this embodiment, an example was described in which the subject disappears from the screen of the monitor 111 because it is no longer the subject being filmed, but this is not limited to this case. For example, even if the camera switches to a tracking camera that is not outputting video to the monitor 111, the person who was watching the subject on the monitor 111 will still need to look for the subject.

[0062] More specifically, consider a scenario where, for example, there is a tracking camera D, which was not present in the configuration described so far, and the video data from tracking camera D is not displayed on monitor 111 due to limitations on the number of display areas that can be shown on monitor 111. In that case, for example, if the tracking camera that was filming the subject switches from tracking camera C109 to tracking camera D, the subject will no longer be displayed on monitor 111, and the person who was watching the subject will have to search for the subject's whereabouts.

[0063] In response to this, when switching to a tracking camera that is not outputting video to the monitor 111, the same processing may be carried out as when the subject is no longer being filmed, as described above. This allows notification that the subject is no longer displayed on the monitor 111, even when switching to a tracking camera that is not outputting video to the monitor 111.

[0064] As described above, in this embodiment, in the multi-camera shooting system, a notification is displayed when the subject being filmed by the tracking camera changes and the subject that was displayed in the image of that tracking camera is no longer displayed. This allows the person viewing the monitor to notice that the subject is no longer displayed, thus avoiding the need to search for the subject.

[0065] (Embodiment 3) Embodiments 1 and 2 described examples of shooting one subject with one tracking camera. This embodiment describes examples of shooting multiple subjects with one tracking camera, such as when the number of subjects is greater than the number of tracking cameras, or when it is desired to shoot multiple compositions.

[0066] An example of the configuration of the multi-camera shooting system according to this embodiment is the same as in Embodiment 1. The operation of the tracking target change detection unit 103, notification display control unit 104, LAN 105, overhead camera 106, tracking camera A 107, tracking camera B 108, tracking camera C 109, video switcher 110, and monitor 111 according to this embodiment is the same as in Embodiment 1.

[0067] In this embodiment, the tracking target determination unit 102, in addition to the processing described in Embodiment 1, detects the distance between subjects and, if the distance between subjects is close (below a threshold), processes to group multiple subjects together as tracking targets. It then instructs each tracking camera to photograph the subject. If the tracking camera has the function to track multiple subjects simultaneously, the tracking target determination unit 102 instructs multiple subjects to be tracked. On the other hand, if the tracking camera does not have the function to track multiple subjects simultaneously, for example, the tracking target determination unit 102 may calculate the center of gravity position (or the center position between subjects) of the subjects to be tracked simultaneously and instruct the tracking camera to point at that position.

[0068] In addition to the processing described in Embodiments 1 and 2, the notification display control unit 104 of this embodiment provides instructions to display an auxiliary information on the monitor 111 when the camera tracking the subject changes during tracking of multiple people, so that the viewer of the monitor 111 does not miss the subject they are focusing on. Specifically, consider, for example, the case where tracking camera B108 switches to tracking multiple people, and the subject that was being tracked by tracking camera C109 is now being tracked by tracking camera B108. In that case, the notification display control unit 104 instructs tracking cameras B108 and C109 to output the video data with an outer frame.

[0069] Figures 8A to 8D illustrate an example of a notification display method for the multi-camera shooting system according to this embodiment.

[0070] Figure 8A shows an example of the screen display on monitor 111 before the subject being tracked by the tracking camera changes. This example shows a case where there are four subjects, with subject D added compared to Figure 4A. The overhead camera 106 shows all four subjects A through D, tracking camera A 107 shows subject A, tracking camera B 108 shows subject B, and tracking camera C 109 shows subject C.

[0071] Figure 8B shows an example of the screen display on monitor 111 after subjects B and C have approached each other, causing tracking camera B108 to track subjects B and C, and tracking camera C109 to track subject D. At this time, the display area of ​​monitor 111 where subject C is displayed moves from the lower left to the upper right, so an outer frame is displayed around the display areas of tracking cameras B and C. This prevents the viewer of monitor 111 from losing sight of subject C.

[0072] Furthermore, for example, as shown in Figure 8C, if the tracking camera tracking the subject is changed, information (a shape or symbol) indicating the display area where the subject will be displayed after the change may be displayed in the display area where the subject was displayed before the change. In Figure 8C, information indicating that the subject C, which was displayed in the display area of ​​the tracking camera C's video data (the lower left display area of ​​the monitor 110), has moved to the upper right display area of ​​the monitor 110 is superimposed on the display area of ​​the tracking camera C's video data. This information is represented by a symbol that is black in the upper right display area and white in the other display areas.

[0073] Furthermore, as shown in Figure 8D, for example, identification information (such as an identification number or identification symbol) may be assigned to each subject in advance, and a map may be displayed that shows which display area each subject is shown in. When the tracking camera tracking a subject changes, the system may notify the user by flashing the map display of the tracking camera currently tracking the subject and the tracking camera that was tracking the subject previously. For example, in Figure 8D, subject C, which was being photographed by tracking camera C109, changes to subject D, and tracking camera B108 is now photographing subject C. At this time, the subject photographed by tracking camera B108 changes from subject B to subject B and subject C. In such a case, the identification information "D" in the subject display map of tracking camera C109 may be flashed, and the identification information "B, C" in the subject display map of tracking camera B108 may also be flashed.

[0074] Furthermore, as explained in Figures 6B and 7C, if the video data from the overhead camera 106 displays frames indicating the field of view of each tracking camera, the frame indicating the field of view of the corresponding tracking camera may be blinked to notify the user when the tracking camera tracking the subject changes.

[0075] Furthermore, the method shown in Figure 8D has the advantage of allowing the raw video data to be saved as is, since there is no need to process the video data captured by each tracking camera.

[0076] In this embodiment, an example has been described where the tracking switches from tracking one person to tracking multiple people, but this is not limited to this case. For example, if the tracking switches from tracking two people to tracking three people, and the number of subjects being tracked increases, the notification should be displayed in the same way. Similarly, for example, if the number of people being tracked remains the same but the subject being tracked changes while tracking multiple people is in progress, the same applies.

[0077] Conversely to the example of this embodiment, if the tracking switches from tracking multiple people to tracking one person and the number of subjects being tracked decreases, notification may be given using the method described in Embodiment 2.

[0078] As explained above, in this embodiment, in a multi-camera shooting system, when one tracking camera can track multiple people, a notification is displayed on the monitor when the camera tracking the subject changes. This prevents the person viewing the monitor from losing sight of the subject.

[0079] According to this disclosure, it becomes possible to suppress the loss of sight of the subject.

[0080] (Other Embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0081] The technical ideas derived from this disclosure are not limited to the exemplary embodiments disclosed, but are intended to encompass various modifications of the exemplary embodiments, or substitutions with equivalent structures or functions. The scope of the following claims should be interpreted in the broadest way to encompass all such modifications and equivalent structures and functions.

[0082] This application claims priority based on Japanese Patent Application No. 2025-012291, filed on 28 January 2025, and all of its contents are incorporated herein by reference.

Claims

1. A control device comprising: detection means for detecting that the subject being photographed by at least one of the subjects being photographed by a plurality of photographing devices has changed; and control means for notifying the display screen, which displays the images taken by each of the plurality of photographing devices, that the image of the photographing device in which the change in the subject being photographed occurred has been detected by the detection means.

2. The control device according to claim 1, characterized in that the detection means detects a change in the subject to be photographed by detecting an exchange between a first subject to be photographed by a first photographing device among the plurality of photographing devices and a second subject to be photographed by a second photographing device among the plurality of photographing devices.

3. The control device according to claim 2, characterized in that, when the swap is detected, the control means provides a notification on the display screen showing the image of the first shooting device and the image of the second shooting device in which the swap occurred.

4. The control device according to claim 3, characterized in that the control means provides the notification by displaying an outer frame on the display area of ​​the image of the first shooting device and the display area of ​​the image of the second shooting device.

5. The control device according to claim 3, characterized in that the control means provides notification by displaying information indicating the position of the display area of ​​the image of the second camera that will display the first subject after the swap in the display area of ​​the image of the first camera that was displaying the first subject before the swap, and by displaying information indicating the position of the display area of ​​the image of the first camera that will display the second subject after the swap in the display area of ​​the image of the second camera that was displaying the second subject before the swap.

6. The control device according to claim 3, wherein the display screen includes an image of an overhead shooting device that captures the entire shooting area, the control means displays information indicating the field of view of each of the plurality of shooting devices on the image of the overhead shooting device, and displays information indicating the position of the display area of ​​the image of each shooting device, and the control means performs the notification by highlighting the information indicating the field of view of the first shooting device and the information indicating the position of the display area of ​​the image of the first shooting device on the display screen, and the information indicating the field of view of the second shooting device and the information indicating the position of the display area of ​​the image of the second shooting device on the display screen.

7. The control device according to claim 2, wherein the display screen includes a map that shows in a list which camera's video display area each subject is displayed in, and the control means, when the swap is detected, notifies by highlighting the positions on the map corresponding to the video display area of ​​the first camera and the video display area of ​​the second camera where the swap occurred.

8. The control device according to claim 7, characterized in that the control means provides notification by flashing identification information of the second subject being photographed by the first shooting device after the swap at a first position on the map corresponding to the display area of ​​the image of the first shooting device where the swap occurred, and by flashing identification information of the first subject being photographed by the second shooting device after the swap at a second position on the map corresponding to the display area of ​​the image of the second shooting device where the swap occurred.

9. The control device according to claim 2, characterized in that the control means displays a frame unique to each subject for the display area of ​​the image of each shooting device on the display screen, and provides notification by highlighting the frames of the respective display areas where the swap has occurred.

10. The control device according to any one of claims 1 to 9, characterized in that the detection means detects a change in the subject being photographed by any of the plurality of photographing devices, by detecting that the subject being photographed by any of the plurality of photographing devices is no longer being photographed by any of the plurality of photographing devices.

11. The control device according to claim 10, characterized in that the control means provides a notification indicating that the subject is no longer being photographed by any of the plurality of photographing devices.

12. The control device according to claim 11, characterized in that the control means provides the notification by displaying an outer frame in the display area of ​​the image of the camera that was photographing the subject.

13. The control device according to claim 11 or 12, characterized in that the control means provides the notification by displaying a message indicating that the subject is no longer being photographed by any of the plurality of photographing devices in the display area of ​​the image of the photographing device that was photographing the subject.

14. The control device according to claim 10, wherein the display screen includes an image of an overhead shooting device that captures the entire shooting area, the control means displays information indicating the field of view of each of the plurality of shooting devices on the image of the overhead shooting device, and displays information indicating the position of the display area of ​​the image of each shooting device on the display screen, and the control means performs the notification by highlighting information indicating the position of the display area on the display screen of the image of the shooting device that was capturing the subject which is no longer captured by any of the plurality of shooting devices, and information indicating the position of the subject which is no longer captured by any of the plurality of shooting devices on the image of the overhead shooting device.

15. The control device according to claim 10, wherein the display screen includes a map that shows in a list which shooting device's video display area each subject is displayed in, and the control means provides the notification by highlighting the position on the map corresponding to the video display area of ​​the shooting device that was shooting the subject which is no longer being shot by any of the plurality of shooting devices.

16. The control device according to any one of claims 1 to 15, characterized in that the detection means detects a change in the subject being photographed by one of the plurality of photographing devices, by detecting that the subject being photographed by one of the photographing devices is now being photographed by another photographing device together with another subject.

17. The control device according to 16, characterized in that the control means provides notification by displaying an outer frame in the display area of ​​the image of the camera that was capturing the subject before the change was detected and in the display area of ​​the image of the other camera that began capturing the subject after the change was detected.

18. The control device according to claim 16, characterized in that the control means provides notification by displaying information indicating the position of the display area of ​​the image of the other shooting device in which the subject is photographed after the change is detected, in the display area of ​​the image of the shooting device in which the subject was photographed before the change was detected.

19. The control device according to 16, wherein the display screen includes a map that shows in a list which camera's video display area each subject is displayed in, and the control means provides the notification by highlighting the position on the map corresponding to the video display area of ​​the camera that was capturing the subject before the change was detected, and the position on the map corresponding to the video display area of ​​the other camera that began capturing the subject after the change was detected.

20. A method for operating a control device, comprising: a detection step of detecting that the subject being photographed by at least one of the subjects being photographed by a plurality of photographing devices has changed; and a control step of, when the change in the subject being photographed is detected by the detection step, notifying the display screen that displays the images taken by each of the plurality of photographing devices to show the image of the photographing device in which the change in the subject being photographed occurred.

21. A program for causing a computer to execute the operation method of the control device described in claim 20.

Images