control device

Abstract

In a control device that controls an imaging device, the desired image can be easily selected. [Solution] The CPU 201 acquires images from multiple cameras 100, and the CPU 201 and the inference unit 205 obtain the positional relationship of multiple subjects from the acquired images. Then, based on the positional relationship of the subjects and the acquired images, the CPU 201 assigns camera numbers to select the multiple cameras 100 in the same order as the arrangement of the multiple subjects, and the operation interface 206 selects one of the multiple cameras 100 based on the camera numbers assigned by the assignment means.

Description

Technical Field

[0001] The present invention relates to a control device for controlling an imaging device.

Background Art

[0002] In recent years, network cameras capable of control and video distribution via a network have become widespread. With this technology, a user can operate a camera controller to remotely control various functions necessary for using a camera, such as control of zoom, focus, etc., image quality setting, power control, status management, network setting, etc. via a network.

[0003] Furthermore, with the spread of such network cameras, an IP remote shooting technology has been proposed that simultaneously controls a plurality of network cameras from a remote location and shoots images from various shooting angles.

[0004] However, since the number of joysticks, dials, and buttons on a camera controller is limited, it is common to register the cameras to be operated in advance in association with camera numbers in the camera controller, and select the cameras to be operated based on those camera numbers and operate them one by one. Therefore, when a plurality of cameras are connected to the camera controller, it is necessary to select and switch the cameras to be operated each time, and perform desired operations on each camera while checking the images of the selected cameras.

[0005] As a result, in IP remote shooting using a camera controller, since a user can only view the images of the cameras to be operated, it is difficult to grasp the overall situation of the shooting site, and there is a problem that it is difficult to switch to the camera shooting at the desired shooting angle.

[0006] A conventional solution to these challenges involves connecting an overhead camera to a camera controller, allowing the user to monitor the overall situation at the shooting location by checking the overhead camera's footage and the footage from other cameras as needed, and then switching to the camera capturing the desired field of view. However, this method requires the user to constantly switch the target camera to the overhead camera to check the situation, leaving the problem that it is difficult to directly switch to the camera capturing the desired field of view.

[0007] Therefore, in IP remote shooting using a camera controller, it is necessary to be able to quickly switch to the camera that is directly capturing the desired angle of view while being able to grasp the overall situation of the shooting site.

[0008] Patent Document 1 describes a video selection device in which the subject position is presented to a video output device, and the desired video is selected by selecting the presented subject position with a subject position input device. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2000-201345 [Overview of the project] [Problems that the invention aims to solve]

[0010] Patent Document 1 describes a method for selecting a camera to capture a desired image by selecting a presented subject position. Therefore, Patent Document 1 requires a subject position presentation device such as a display device. However, some camera controllers mentioned above allow switching cameras by operating buttons labeled with numbers such as camera numbers. When switching cameras using such button operations, the method of selecting the subject position on a screen, as in Patent Document 1, is difficult to apply. Furthermore, when multiple cameras are capturing the same subject, in Patent Document 1, since the subject positions are the same, only one subject position (push button) is displayed on the subject position presentation device. Therefore, for example, if there are cameras capturing only the upper body and cameras capturing the whole body of the same subject, it is not possible to distinguish between them and switch between them.

[0011] This invention was made in view of the above problems, and aims to allow for easy selection of desired images. [Means for solving the problem]

[0012] To solve the above problems, the control device of the present invention is characterized by comprising: an image acquisition means for acquiring captured images from a plurality of imaging devices; a subject position acquisition means for acquiring the positional relationship of a plurality of subjects from the captured images acquired by the image acquisition means; an assignment means for assigning identification codes to select the plurality of imaging devices in the same order as the arrangement of the plurality of subjects, based on the positional relationship acquired by the subject position acquisition means and the captured images; and a selection means for selecting one of the plurality of imaging devices based on the identification codes assigned by the assignment means. [Effects of the Invention]

[0013] According to the present invention, identification codes are assigned in the same order as the arrangement of multiple subjects, based on the positional relationship of the subjects and the captured image, so that the desired image can be easily selected. [Brief explanation of the drawing]

[0014] [Figure 1] It is a diagram showing a configuration example of a system including a control device according to the first embodiment. [Figure 2] It is a block diagram of the camera and the controller shown in FIG. 1. [Figure 3] It is a diagram showing an example of connection camera information. [Figure 4] It is an external view showing an example of the housing of the controller shown in FIG. 1. [Figure 5] It is a flowchart showing the control process of the controller shown in FIG. 1. [Figure 6] It is a diagram showing an example of the arrangement relationship of the cameras. [Figure 7] It is a diagram showing an example of the captured image of the camera. [Figure 8] It is a diagram showing an example of the inferred similarity. [Figure 9] It is a diagram showing an example of the updated subject position. [Figure 10] It is a diagram showing an example of the relationship between the camera number after assignment and the camera selection button. [Figure 11] It is a flowchart showing the control process of the controller according to the second embodiment. [Figure 12] It is a diagram showing an example of the arrangement relationship of the cameras. [Figure 13] It is a diagram showing an example of the captured image of the camera. [Figure 14] It is a diagram showing an example of a camera number group. [Figure 15] It is a diagram showing an example of the relationship between the camera number after assignment and the camera selection button. [Figure 16] It is a flowchart showing the control process of the controller according to the third embodiment. [Figure 17] It is a diagram showing an example of a warning display. [Figure 18] It is a flowchart showing the control process of the controller according to the fourth embodiment. [Figure 19] It is a diagram showing an example of the arrangement relationship of the cameras. [Figure 20] It is a diagram showing an example of the captured image of the camera. [Figure 21] This figure shows an example of the similarity of subjects. [Figure 22] This figure shows an example of determining the relative positions of cameras. [Modes for carrying out the invention]

[0015] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0016] <First Embodiment> Figure 1 shows an example of the system configuration of this embodiment. This system has five cameras 100a to 100e and a controller 200 for controlling the cameras 100a to 100e. The controller 200 is also called a camera controller and is a control device according to this embodiment. The cameras 100a to 100e and the controller 200 are connected via a network 300 so that they can communicate with each other.

[0017] The controller 200 can control the operation of cameras 100a to 100e and acquire information by sending commands to cameras 100a to 100e based on a camera control communication protocol.

[0018] Camera 100a is a camera for photographing subjects, and camera 100e is installed as an overhead camera for understanding the overall situation of the shooting site. In other words, cameras 100a to 100e are imaging devices according to this embodiment, and camera 100e functions as an overhead imaging device for capturing overhead images.

[0019] In this embodiment, a total of five cameras are connected: cameras 100a to 100d for photographing subjects and camera 100e for overhead shots. However, the number and type of cameras are not limited as long as the device supports the camera control communication protocol. For example, it could be a peripheral device such as a tripod head without imaging capabilities, a camera with a PTZ (Panoramac Tilt Zoom) mechanism, or a camera with only zoom capabilities.

[0020] Figure 2 shows the main configuration of the cameras 100a to 100e and controller 200 that make up the system shown in Figure 1. Since cameras 100a to 100e have the same configuration, they are shown as camera 100 in Figure 2. Furthermore, hereafter, when cameras 100a to 100e are not distinguished, they will simply be referred to as camera 100.

[0021] [Camera 100 Configuration] The camera 100 of this embodiment includes a CPU 101, a RAM 102, a ROM 103, an imaging unit 104, and a communication unit 105. These units are connected by an internal bus 106. Each unit is electrically powered by power obtained by rectifying AC power supplied from an external source to a predetermined voltage, or by power supplied from a built-in battery (not shown).

[0022] The CPU 101 is a system control unit that controls the entire system of the camera 100. The CPU 101 performs control and calculation processing of each block by loading the control program recorded in the ROM 103 into the RAM 102 and executing it.

[0023] RAM102 is used as work memory to temporarily store control programs and data.

[0024] ROM103 consists of non-volatile storage devices such as flash memory, HDDs (Hard Disk Drives), SSDs (Solid State Drives), and SD cards. ROM103 is used as a persistent storage area for the OS (Operating System), various programs, and various data, as well as for short-term data storage.

[0025] The imaging unit 104 receives the light formed through the lens on the image sensor, converts the received light into an electric charge, and acquires a moving image. For example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be used as the image sensor. Alternatively, a CCD (Charge Coupled Device) image sensor may also be used.

[0026] The communication unit 105 is for network communication with external devices. In this embodiment, the communication unit 105 is used for communication with the controller 200.

[0027] [Controller 200 Configuration] Next, the controller 200 will be described. The controller 200 in this embodiment includes a CPU 201, RAM 202, ROM 203, communication unit 204, inference unit 205, operation I / F 206, and display unit 207. Each of the above-mentioned blocks is connected by an internal bus 208.

[0028] The CPU 201 is the system control unit that controls the entire system of the controller 200. The CPU 201 executes the program stored in the ROM 203 by loading it into the RAM 202, thereby controlling each block and performing calculations, and executing the flowchart described later. The CPU 201 also manages the connected camera 100 and is responsible for storing connected camera information such as the camera group, camera number, IP address of camera 100, and camera name in the ROM 203. An example of connected camera information is shown in Figure 3.

[0029] In Figure 3, camera names A to D correspond to one of cameras 100a to 100d, and camera name E corresponds to camera 100e. Camera A has camera number 1, camera B has camera number 2, and camera C has camera number 3. Camera D has camera number 4, and camera E has camera number 5. Cameras A to E (camera numbers 1 to 5) are all registered to camera group "1".

[0030] The camera group, camera number, and camera name can be arbitrarily set by the user. Alternatively, the camera group and camera number can be automatically set by a process described later. In this embodiment, the camera number is the number used when selecting cameras A to E, and is the identification code for cameras A to E (cameras 100a to 100e). The identification code is not limited to numbers (digits); it may also be letters or other characters with a defined sequence. A camera group is a grouping of multiple cameras 100, and for example, up to the number of buttons on the camera selection button 401 (described later) can be registered in one group.

[0031] Furthermore, the CPU 201 displays a menu screen on the display unit 207, and when it receives an operation according to this menu screen via the operation interface 206, it performs settings on the controller 200 itself and controls the connected camera 100.

[0032] RAM202 is used as work memory to temporarily store control programs and data.

[0033] ROM203 consists of non-volatile storage devices such as flash memory, HDDs, SSDs, and SD cards. ROM203 is used as a persistent storage area for the OS, various programs, and various data, as well as for short-term data storage.

[0034] The communication unit 204 is for network communication with external devices. In this embodiment, the communication unit 204 is used for communication with the camera 100.

[0035] The inference unit 205 estimates the presence or absence and position of a predetermined object from the captured image captured by the camera 100 and received by the aforementioned communication unit 204, etc. The inference unit 205 is composed of a computing device specialized for image processing and inference processing, such as a so-called GPU (Graphics Processing Unit). While a GPU is generally effective for inference processing, equivalent functionality may be achieved with a reconfigurable logic circuit such as an FPGA (Field-Programmable Gate Array). Alternatively, the processing of the inference unit 205 may be handled by the CPU 201.

[0036] The operation interface 206 is an interface (I / F) for receiving user input to the controller 200. The operation interface 206 includes multiple buttons, joysticks, and other operating components mounted on the controller 200's casing, as well as a GUI (Graphical User Interface) displayed on the display unit 207. When these operating components or the GUI are operated, the operation information is input to the CPU 201. This allows for the configuration of the controller 200 itself and the control of the camera 100 connected to the controller 200. A detailed explanation will follow later.

[0037] The display unit 207 is composed of, for example, a liquid crystal panel and a backlight, and displays captured images acquired from the camera 100, setting screens, etc. In this embodiment, the display unit 207 displays the captured images taken by the camera 100 and the settings of the camera 100. Although this embodiment shows an example in which the controller 200 has a display unit 207, the configuration is not limited to this. For example, the display device having the function of the display unit 207 and the controller 200 may each be configured in separate housings.

[0038] [How to operate the control components] Figure 4 shows an example of the housing (external appearance) of the controller 200 according to this embodiment. In addition to the display unit 207, the housing of the controller 200 is provided with a camera selection button 401, a camera group switching button 402, a zoom seesaw 403, and a joystick 404, which are included in the operation I / F 206. Note that the configuration of the controller 200 in this embodiment is not limited to this, and for example, the camera selection button 401 may be a GUI displayed on the display unit 207.

[0039] The camera selection buttons 401 are buttons located on the casing of the controller 200, and each button is assigned a number from 1 to 10. For example, when the display unit 207 is set to a screen for switching the camera 100 to be controlled, pressing the "1" button on the camera selection buttons 401 will select camera 100, which is assigned camera number "1", as the target for control. Camera 100, which is the camera number selected as the target for control, can then be operated using the zoom seesaw 403, joystick 404, etc.

[0040] The camera group switching button 402 is a button located on the housing of the controller 200. By pressing the camera group switching button 402, the camera group can be toggled between camera groups 1 to 20. For example, if camera 100, with camera number "1", is selected as the control target, and the camera group switching button 402 is operated to change it to "2", which indicates the second group, camera 100, with camera number "11", will be selected. In this embodiment, the camera group switching button 402 is described as toggling the camera group, but this is not the only way. For example, a GUI could be used in which the camera group destination is displayed on the display unit 207 after the camera group button is pressed.

[0041] The zoom seesaw 403 is mounted on the casing of the controller 200. By operating the zoom seesaw 403, the zoom of the camera 100 being controlled can be controlled.

[0042] The joystick 403 is mounted on the casing of the controller 200. By operating the joystick 403 in the left, right, up, and down directions, the pan and tilt of the camera 100 or peripheral devices being controlled can be controlled.

[0043] [Operation of this embodiment] Next, the operation of the controller 200 with the above configuration, specifically the operation of assigning camera numbers according to the relative positions of subjects (control method of the control device), will be explained using the flowchart in Figure 5. This flowchart is realized by the CPU 201 reading the program stored in the ROM 203, loading it into the RAM 202, and executing control and calculation processing for each part. Furthermore, the flowchart in Figure 5 is executed by the CPU 201 acquiring information based on user operations (operation information) via the operation I / F 206 and performing actions based on this operation information. Note that the processing shown in Figure 5 is merely an example, and the order is not limited.

[0044] Furthermore, in the following explanation, it is assumed that five cameras 100, designated A to E, have already been registered sequentially to camera numbers 1 to 5 of the controller 200. Cameras A to D are cameras used for photographing subjects, while camera E is installed as an overhead camera to capture an overall view of the shooting site, including the subjects being photographed by cameras A to D, in order to understand the situation of the entire shooting location. In this embodiment, the subject is described as a living being such as a person, but any object that can be recognized separately from the background is acceptable.

[0045] Figure 6 shows an example of the camera arrangement in this embodiment. In the example in Figure 6, looking from the camera side towards the subject side, the cameras are arranged from left to right in the order of Camera A, Camera B, Camera E, Camera C, and Camera D. Furthermore, each camera is positioned to capture images of at least one of the four subjects: Subject A, Subject B, Subject C, and Subject D.

[0046] Figure 7 shows an example of an image captured by each camera. Camera A is positioned to photograph subject A, camera B to photograph subject B, camera C to photograph subject C, and camera D to photograph subject D as the main subject, while camera E is positioned so that all subjects fit within the field of view. In other words, in the examples shown in Figures 6 and 7, the camera positions and subject positions intersect. Therefore, if the camera selection button 401 is registered in the order of the arrangement of cameras 100, it will not match the order of the subjects.

[0047] The following describes each step in the flowchart of Figure 5. Furthermore, the notation for each step is omitted by prefixing it with "S". First, in S501, the CPU 201 receives (acquires) captured images (image data) from the camera 100 registered with the controller 200 via the communication unit 204 and stores them in the RAM 202. In other words, the CPU 201 functions as an image acquisition means that acquires captured images from multiple imaging devices.

[0048] In this embodiment, captured images are received via the communication unit 204, but this is not limited to this. For example, a separate image input unit may be provided, and captured images may be received by connecting it via SDI (Serial Digital Interface) or HDMI (High-Definition Multimedia Interface).

[0049] Next, in S502, the CPU 201 reads the captured image stored in RAM 202 in S501 and inputs it to the inference unit 205. Based on the input captured image, the inference unit 205 infers subject information such as the position of the subject in the captured image (subject position) and the size of the subject in the captured image (subject size), and stores it in RAM 202. The inference unit 205 has a trained model created using machine learning such as deep learning, receives an image as input data, and outputs information such as the type of subject (e.g., person), subject size, and subject position as output data. If multiple images are input, it also outputs the similarity of the subjects contained in each image.

[0050] In this embodiment, the subject position is described as the coordinates indicating the centroid of the subject within the image. The subject size is described as the height and width (e.g., number of pixels) of the frame surrounding the detected subject within the image. The subject similarity is a value from 0 to 100, and a higher value indicates that the subjects in the input image are more similar.

[0051] Furthermore, subject information is associated with each subject in the captured image and stored in RAM202. For example, in the example in Figure 7, the image captured by camera A contains only subject A, so only the subject information of subject A is associated with and stored in that image. On the other hand, the image captured by camera E contains subjects A, B, C, and D, so the subject information of all four subjects, A, B, C, and D, is associated with and stored in that image.

[0052] Returning to the explanation of Figure 5, in S503, the CPU 201 determines whether the inference in S502 has been completed for all the captured images stored in RAM 202 in S501. If it is determined that it is completed (S503:Yes), it proceeds to S504; if it is determined that it is not yet completed (S503:No), it proceeds to S502.

[0053] Next, in S504, the CPU 201 reads the captured image stored in S501 and the subject information stored in S502 from the RAM 202, and determines that the captured image with the most associated subject information is the image captured by the overhead camera to grasp the overall situation of the shooting site. In other words, the CPU 201 uses the captured image with the most subjects among the acquired captured images as the overhead image (reference image), and executes the processes in S505 and S506 described later to obtain the positional relationships.

[0054] The CPU 201 stores the camera number associated with the image captured by the determined overhead camera in the RAM 202 as the overhead camera number. In the example in Figure 7, camera E, which has four pieces of subject information associated with it, is determined to be the overhead camera, and camera number 5 is stored in the RAM 202 as the overhead camera.

[0055] If there are multiple cameras 100 with the same number of subject information, the one with the smallest average subject size may be determined to be the overhead camera. That is, the CPU 201 may use the image with the largest number of subjects and the smallest average subject size among the acquired images as the reference image, and use this reference image to perform the processes S505 and S506 described later to obtain the positional relationship.

[0056] Furthermore, in this embodiment, the system receives captured images from the camera 100 and performs overhead camera determination based on the number of subject information points (number of subjects) inferred from the captured images, but it is not limited to this. For example, the user may set the overhead camera via the operation I / F 206, or inference may be performed based on image data in which subjects have been captured and stored in the ROM 203 in advance. That is, the ROM 203 may function as a storage means in which reference images for acquiring positional relationships are stored in advance, and the CPU 201 may acquire the positional relationships by performing the processing in S505 and S506 described later using the acquired captured images and the reference images stored in the storage means.

[0057] Next, in S505, the CPU 201 reads the subject information stored in S502 from the RAM 202 and determines the main subject of each camera 100 based on that subject information. The main subject is the primary subject among the subjects captured by the camera 100, as described above. In other words, the CPU 201 functions as a main subject determination means that determines the primary subject for each acquired image. The CPU 201 stores the subject information of the determined main subject in the RAM 202. In determining the main subject, if there is only one subject information, that subject is determined to be the main subject; if there are multiple subject information, the subject with the largest subject size is determined to be the main subject.

[0058] Next, in S506, the CPU 201 calculates the subject position. First, the CPU 201 reads the captured image stored in S501 from the RAM 202 and inputs it to the inference unit 205. The inference unit 205 reads the overhead camera number stored in S504 from the RAM 202 and infers the similarity between all subjects on the captured image associated with the camera number of the overhead camera and the main subject on the other captured images. For example, in the example in Figure 7, the similarity between subjects A to D on the captured image of camera E and the main subject A of camera A, the main subject B of camera B, the main subject C of camera C, and the main subject D of camera D is inferred. Figure 8 shows an example of the inferred subject similarity.

[0059] In Figure 8, subject A on camera E has a high degree of similarity to the main subject A on camera A, but a low degree of similarity to the main subjects on cameras B through D. Subject B on camera E has a high degree of similarity to the main subject B on camera B, but a low degree of similarity to the main subjects on cameras A, C, and D. Subject C on camera E has a high degree of similarity to the main subject C on camera C, but a low degree of similarity to the main subjects on cameras A, B, and D. Subject D on camera E has a high degree of similarity to the main subject D on camera D, but a low degree of similarity to the main subjects on cameras A through C.

[0060] The inference unit 205 reads subject information from the RAM 202 and, for subjects estimated to have a similarity of, for example, 50 or more, overwrites the subject position with the subject position on the image captured by the overhead camera and stores it in the RAM 202. Figure 9 shows an example of the updated subject position. For the main subject being photographed by cameras A to D, the subject position in the coordinate system of camera E, which is determined to be the overhead camera, has been updated accordingly.

[0061] In this way, the positional relationships of multiple subjects A to D are calculated and obtained. That is, CPU 201 functions as a subject position acquisition means that obtains the positional relationships of multiple subjects from the acquired captured image. Furthermore, CPU 201 obtains the positional relationships using an overhead image.

[0062] Next, in S507, the CPU 201 associates the subject position with the camera number. The CPU 201 reads the subject position and its associated camera number stored in S506 from the RAM 202. The CPU 201 reassigns the camera numbers from the read subject positions so that the values ​​indicating the x-coordinate are in ascending order, and stores them in the RAM 202. In other words, the CPU 201 functions as an assignment means that assigns identification codes to select multiple imaging devices in the same order as the arrangement of multiple subjects, based on the positional relationship of the subjects and the captured image.

[0063] Finally, CPU201 changes the camera number of the camera identified as an overhead camera in S504 to the last number in the camera group and registers it.

[0064] As explained above, S501 functions as the image acquisition process, S506 functions as the subject position acquisition process, and S507 functions as the assignment process.

[0065] Figure 10 shows the reassigned camera numbers and an example of assigning the camera numbers after applying this embodiment to the camera selection button 401 shown in Figure 4. Because the subject position was overwritten in S506, the x-coordinates of the subject positions are, in order from closest to the origin, Camera C, Camera D, Camera A, and Camera B. Therefore, as shown in the table in the middle of Figure 10, Camera C is assigned camera number "1", Camera D is assigned camera number "2", Camera A is assigned camera number "3", and Camera B is assigned camera number "4". Camera E is assigned camera number "10", which is the last in the camera group. Therefore, cameras A through E can be switched by selecting the button on the camera selection button 401. In other words, the operation I / F 206 functions as a selection means that selects one of the multiple imaging devices based on the identification code assigned by the assignment means.

[0066] According to this embodiment, camera numbers can be assigned to the controller 200 in the order of the positions of the subjects being photographed by the registered cameras 100. This makes it possible to select a camera without having to check each captured image, even in cases where the camera position and subject position intersect, such as as shown in Figure 6, where the camera installation order does not match the actual order of the captured subjects. Therefore, it is possible to easily select the desired camera when operating the camera using pre-arranged buttons such as the camera selection button 401 on the controller 200. Consequently, the desired image can be easily selected.

[0067] <Second Embodiment> Next, a control device according to the second embodiment will be described. Note that the same configuration as the first embodiment will not be described, and the following description will focus on the differences from the first embodiment.

[0068] This embodiment describes a function that groups cameras based on specific conditions and assigns camera numbers based on the location information of the group and the subject.

[0069] The configuration of the controller 200 according to this embodiment is the same as that of the first embodiment, so a description will be omitted. The differences from the first embodiment will be explained below using the flowchart in Figure 11. This flowchart is realized when the CPU 201 reads the program stored in the ROM 203, loads it into the RAM 202, and executes control and calculation processing for each part. Note that the processing shown in Figure 11 is merely an example, and the order is not limited.

[0070] Furthermore, in this embodiment, it is assumed that five cameras 100, designated A to E, are already registered sequentially to camera numbers 1 to 5 of the controller 200. Cameras A to D are cameras used for photographing subjects, while camera E is installed as an overhead camera to capture an overall view of the shooting site, including the subjects being photographed by cameras A to D, in order to understand the situation of the entire shooting location.

[0071] Figure 12 shows an example of the camera arrangement in this embodiment. In the example in Figure 12, looking from the camera side towards the subject side, the cameras are arranged from left to right in the order of Camera A, Camera B, Camera E, Camera C, and Camera D. Furthermore, each camera is positioned to capture images of at least one of the three subjects, Subject A, Subject B, and Subject C.

[0072] Figure 13 shows an example of an image captured by each camera. Camera A is positioned to capture subject A, camera B to capture subject B, camera C to capture subject C, and camera D to capture subject C as the main subject. Camera E is positioned so that all subjects fit within its field of view.

[0073] The following describes each step of the flowchart in Figure 11. Steps S1101 to S1106 perform the same processing as steps S501 to S506 in Figure 5. In step S1107, which follows from S1106, the CPU 201 groups the camera numbers. The CPU 201 reads the subject information of the main subject for cameras other than the overhead camera determined in S1104 from the RAM 202. Based on the read subject information, the CPU 201 associates cameras 100 with the same subject position as cameras 100 that are imaging the same subject, and determines that they belong to the same camera number group. A camera number group refers to a group of cameras 100 that are imaging the same subject, and is different from the camera groups shown in Figure 3, etc. In other words, the CPU 201 functions as a classification means that groups imaging devices with the same main subject into the same group.

[0074] Furthermore, the CPU 201 determines that each camera 100 with a single subject position is an independent camera number group and stores it in the RAM 202. In this embodiment, cameras 100 with the same subject position are described as belonging to the same camera number group, but this is not limited to this. For example, the user may set which cameras 100 belong to which camera number groups via the operation I / F 206.

[0075] Alternatively, information that associates a specific camera with another camera may be stored in the ROM 203 as linked camera information, and the CPU 201 may read the linked camera information to set the camera number group. A specific camera includes not only a camera like camera 100, but also peripheral devices that work in conjunction with the camera, such as a tripod head or slider. In other words, the linked camera information becomes correspondence information that indicates the correspondence relationship with peripheral devices attached to the imaging device, and the CPU 201 functions as a means of acquiring correspondence information, and groups the imaging device and its corresponding peripheral devices into the same group based on the acquired correspondence information.

[0076] In the example shown in Figure 13, cameras C and D, which are both photographing the same subject C, are determined to belong to the same camera number group, while cameras A and B are determined to belong to different camera number groups.

[0077] Figure 14 shows an example of the camera number groups assigned to cameras A through E after executing S1107. As shown in Figure 14, camera A belongs to camera number group "1", camera B belongs to camera number group "2", and cameras C and D belong to camera number group "3". Camera E is an overhead camera and is therefore not included in the grouping.

[0078] Next, in S1108, the CPU 201 associates the subject position with the camera number. The CPU 201 reads the subject information and the camera number group stored in S1107 from the RAM 202. The CPU 201 determines the priority of assigning camera numbers to cameras 100 registered in the same camera number group, based on the subject size of the main subject. In this embodiment, values ​​from 1 to 10 are set in order from the largest subject size of the main subject, with smaller values ​​indicating higher priority. For example, in Figure 14, camera C has a priority of "1" and camera D has a priority of "2". In other words, the CPU 201 functions as a priority setting means that sets priority according to the size of the subject in the captured image captured by imaging devices grouped in the same group.

[0079] The CPU 201 reassigns the camera numbers to the camera 100 that has the highest priority among the cameras 100 registered in the same camera number group, so that the values ​​indicating the x-coordinate of the subject's position are in ascending order. For example, in Figure 13, the cameras 100 that have the highest priority are cameras A, B, and C, and sorting the values ​​indicating the x-coordinate of the subject's position in ascending order results in camera C, camera A, and camera B, as shown in Figure 12.

[0080] The CPU 201 then assigns camera numbers to all cameras 100 registered in the same camera number group except for the highest priority camera 100, based on the button position of the camera selection button 401. For example, let's consider the case where the camera selection buttons 401 are arranged on the chassis in m rows and n columns (where m and n are integers greater than or equal to 2) (2 rows and 5 columns in Figure 4). In this case, cameras 100 belonging to the same camera number group are assigned camera numbers such that the horizontal position of the camera selection button 401 is the same, but the vertical position of the button differs according to priority.

[0081] The horizontal position of the camera selection button 401 refers to the position information indicating which button from the left it is among the multiple buttons that make up the camera selection button 401 installed on the housing. In other words, the horizontal position indicates the column position within an m row and n column arrangement. The vertical position of the camera selection button 401 refers to the position information indicating which button from the top it is among the multiple buttons that make up the camera selection button 401. In other words, the vertical position indicates the row position within an m row and n column arrangement. In this embodiment, if cameras belong to the same camera number group, cameras 100 with higher priority are registered higher up in the button arrangement, and as priority decreases, cameras are registered lower down the button arrangement in order.

[0082] Figure 15 shows an example of assigning the camera numbers after applying this embodiment to the camera selection button 401 shown in Figure 4. As described above, when cameras A, B, and C are arranged in ascending order of the x-coordinate value of the subject position, the order is camera C, camera A, and camera B. Therefore, camera C becomes camera number "1", camera A becomes camera number "2", and camera B becomes camera number "3", and are assigned to the corresponding buttons on the camera selection button 401. Camera D is in the same camera number group as camera C and has a lower priority than camera C, so it is assigned to the button that represents "6", which has the same horizontal position as the button that represents camera number "1" for camera C and is vertically below "1". In other words, camera C becomes camera number "6". This button that represents "6" is in the same column as "1", but in a different row (one row below).

[0083] Since both Camera A and Camera B belong to only one camera number group, there is no Camera 100 that can be assigned to the buttons indicating "7" or "8" below "2" or "3" on the Camera Selection Button 401. In this way, the CPU 201 assigns identification codes based on the relative positions of the subjects, the grouping results, and the priority.

[0084] In this embodiment, it has been explained that camera numbers are assigned to cameras other than the highest priority based on the button position of the camera selection button 401, but this is not limited to this. For example, the user may set the camera number via the operation I / F 206, or the camera number may be set by storing information specifying the camera numbers to be assigned in order of priority in the ROM 203 and reading that information from the CPU 201. Also, if it is not possible to assign a camera number within the same camera group, the camera number may be set in the next camera group.

[0085] Finally, the CPU 201 changes the camera number of the camera identified as an overhead camera in S504 to the last number in the camera group and registers it. In Figure 15, camera E, which is an overhead camera, is registered as camera number "10" in the controller 200 of this embodiment, which is the last number in the camera group.

[0086] As described above, according to this embodiment, in addition to the first embodiment, even when multiple cameras are capturing the same subject, it is possible to easily select the camera you want to operate. Therefore, even when multiple cameras are capturing the same subject, they can be distinguished and switched between, and the desired image can be easily selected.

[0087] <Third Embodiment> Next, a control device according to the third embodiment will be described. Note that the same configuration as in the first and second embodiments will be omitted from the description, and the following description will focus on the differences from the first and second embodiments.

[0088] In this embodiment, in addition to the operation of the first embodiment, the operation when the subject moves will be described. The configuration of the controller 200 according to this embodiment is the same as in the first embodiment, so the description will be omitted. The differences from the first embodiment will be explained below using the flowchart in Figure 16. This flowchart is realized by the CPU 201 reading the program stored in the ROM 203, loading it into the RAM 202, and executing control and calculation processing for each part. Note that the processing shown in Figure 16 is merely an example, and the order is not limited.

[0089] Furthermore, in this embodiment, it is assumed that five cameras, A to E, are already registered sequentially to camera numbers 1 to 5 of the controller 200. Cameras A to D are cameras for photographing subjects, while camera E is installed as an overhead camera to capture an overview of the entire shooting site, including the subjects being photographed by cameras A to D, in order to understand the overall situation of the shooting site. The arrangement of each camera is the same as in Figure 6, and the image data from each camera is the same as in Figure 7.

[0090] The following describes each step in the flowchart shown in Figure 16. First, in S1601, the CPU 201 determines whether to continue processing. The CPU 201 checks whether it has received an instruction to terminate this flowchart via the communication unit 204 or the operation interface 206. If it has not received an instruction to terminate, i.e., to continue (S1601: Yes), it proceeds to S1602. On the other hand, if it has received an instruction to terminate (S1601: No), it terminates this flowchart.

[0091] S1602 to S1607 perform the same processing as S501 to S506 in Figure 5. In S1608, which follows from S1607, the CPU 201 determines whether a camera number change is necessary. The CPU 201 reads the subject position and its associated camera number stored in S1607 from RAM 202. The CPU 201 reassigns the camera numbers from the read subject positions so that the values ​​indicating the x-coordinate are in ascending order.

[0092] The CPU 201 determines whether the camera numbers read from RAM 202 are in the same order as the reassigned camera numbers. If it determines that the camera numbers are in the same order (S1608: No), it determines that no change to the camera numbers is necessary and proceeds to S1601. On the other hand, if it determines that the camera numbers are in the same order (S1608: Yes), it determines that a change to the camera numbers is necessary and proceeds to S1609. When it is determined that a change to the camera numbers is necessary, the CPU 201 determines that the subject has moved. Therefore, the CPU 201 functions as a movement detection means for determining the movement of the subject.

[0093] In this embodiment, it was explained that the system automatically transitions to S1609 when it is determined that the order of the camera numbers is different, but this is not limited to this. For example, the CPU 201 may display a warning on the display unit 207 to the user that the camera order will be changed and to ask whether or not to proceed with the order change, and then decide whether to transition to S1601 or S1609 based on the response to that warning.

[0094] Figure 17 shows an example of a warning displayed on the display unit 207 when the order of the subject positions changes due to the movement of subject C and subject D. If the user selects "Yes" via the operation interface 206, the process proceeds to S1609. On the other hand, if the user selects "No" via the operation interface 206, the process proceeds to S1601 (or the flowchart may be terminated). Note that the warning shown in Figure 17 may simply be a notification, as long as it informs the user that the subject has moved. In other words, the display unit 207 functions as a notification means to notify the user when it determines that the subject has moved, and the operation interface 206 functions as an operation information acquisition means to acquire operation information based on the user's operation.

[0095] S1609, which follows from S1608, performs the same processing as S507. That is, if the CPU 201 determines that the subject has moved, it reacquires the positional relationship of the subject, and if it determines that the positional relationship has changed before and after the reacquisition, it reassigns the identification code. In addition, if a display (notification) as shown in Figure 17 has occurred, the identification code is assigned based on the operation information after notification by the notification means.

[0096] As described above, according to this embodiment, even if the order of the camera numbers changes due to the movement of the subject, the change can be detected and the camera numbers can be reassigned to match the changed order.

[0097] <Fourth Embodiment> Next, the control device according to the fourth embodiment will be described. Note that the same configuration as in the first to third embodiments will be omitted from the description, and the following description will focus on the differences from the first to third embodiments.

[0098] In this embodiment, we will describe a function that uses a shooting camera as a substitute for an overhead camera when a camera equivalent to the overhead camera in the first embodiment does not exist.

[0099] The configuration of the controller 200 according to this embodiment is the same as that of the first embodiment, so a description will be omitted. The differences from the first embodiment will be explained below using the flowchart in Figure 18. This flowchart is realized when the CPU 201 reads the program stored in the ROM 203, loads it into the RAM 202, and executes control and calculation processing for each part. Note that the processing shown in Figure 18 is merely an example, and the order is not limited.

[0100] Furthermore, in this embodiment, it is assumed that four cameras, A to D, are already registered sequentially to camera numbers 1 to 4 of the controller 200. Cameras A to D are cameras used for photographing subjects. In addition, each camera is assumed to have a PTZ (Panoramac Tilt Zoom) mechanism. Note that cameras with a PTZ mechanism are not limited to cameras with a PTZ mechanism on their own, but also include cameras to which peripheral devices with either Panoramac or Tilt functions are attached. In other words, cameras with a PTZ mechanism are variable imaging devices that can change the imaging range.

[0101] Figure 19 shows the arrangement of each camera. Looking from the camera side towards the subject side, the cameras are positioned from left to right in the order of Camera A, Camera B, Camera C, and Camera D. Furthermore, each camera is positioned to capture images of at least one of the three subjects: Subject A, Subject B, and Subject C.

[0102] Figure 20 shows an example of an image captured by each camera. Camera A is positioned to capture subject A as the main subject, camera B as subject B, camera C as subject C, and camera D as subject D, respectively. In addition, subject B is captured by camera A, and subject A is captured by camera B, respectively, as sub-subjects. A sub-subject refers to any subject other than the main subject.

[0103] The following describes each step in the flowchart in Figure 18. Steps S1801 to S1803 perform the same processing as steps S501 to S503 in Figure 5. Step S1804 performs the same processing as step S505 in Figure 5.

[0104] In S1805, which follows from S1804, the CPU 201 determines whether there is an overhead camera. The CPU 201 reads the imaging information stored in S1801 and the subject information stored in S1802 from the RAM 202. The CPU 201 determines that the image with the most associated subject information is the image captured by the overhead camera for understanding the overall situation of the shooting site, and inputs it to the inference unit 205. In this embodiment, it has been explained that the overhead camera determination is made based on the number of subject information (number of subjects) inferred from the image captured by the camera, but this is not limited to this. For example, the user may set the overhead camera via the operation I / F 206, or the inference may be made based on image data stored in the ROM 203 in advance. Also, if there are multiple cameras with the same number of subject information, the one with the smallest average subject size may be determined to be the overhead camera.

[0105] The inference unit 205 infers the similarity between all subjects in the overhead camera's captured image input from the CPU 201 and the main subjects in the images captured by cameras other than the overhead camera, and stores this in the RAM 202. For example, in the example in Figure 20, since camera A and camera B have the most subject information and the same number of subjects, camera B, which has a smaller average subject size, is determined to be the overhead camera, and the image captured by camera B is input to the inference unit 205. The inference unit 205 infers the similarity between subjects A and B in camera B's captured image and subjects A, C, and D, which are the main subjects of cameras A, C, and D.

[0106] CPU201 reads the inference results from RAM202 and determines whether there are any subjects with a similarity of 50 or less to all the main subjects of each camera in the image captured by the overhead camera. If CPU201 determines that there are no subjects with a similarity of 50 or less, it proceeds to S1806 (S1805: Yes). The absence of main subjects with a similarity of 50 or less means that all main subjects are included in the image captured by the camera determined to be the overhead camera, and the overall situation of the shooting site can be understood. Therefore, the overhead camera is found to be present.

[0107] On the other hand, if it is determined that there are subjects with a similarity of 50 or less, the system proceeds to S1807 (S1805: No). The presence of main subjects with a similarity of 50 or less means that not all main subjects are included in the images captured by the camera determined to be an overhead camera, and therefore the overall situation of the shooting location cannot be grasped. For this reason, a camera that was determined to be an overhead camera based on the average size of subjects is ultimately determined not to be an overhead camera.

[0108] Figure 21 shows an example of the similarity between subjects A and B in the images captured by camera B and the main subjects of each camera. In Figure 21, the main subject A of camera A has a similarity above the threshold (70) to subject A on camera B, but the main subject C of camera C and the main subject D of camera D have a similarity below the threshold (10) to both subjects A and B on camera B. Therefore, it is determined that there are subjects with a similarity of 50 or less, and the process proceeds to S1807.

[0109] Returning to the explanation of Figure 18, S1806 performs the same processing as S506 in Figure 5. In S1807, the CPU 201 determines whether the positional relationship of the subjects can be determined. The CPU 201 reads the captured image stored in S1801, the subject information stored in S1802, and the main subject information stored in S1804 from the RAM 202. The CPU 201 inputs the captured image (hereinafter referred to as the input image) of the camera 100 with the lowest camera number among the cameras 100 currently assigned camera numbers to the inference unit 205. The inference unit 205 infers the similarity between subjects that are not the main subject of the input image and the main subjects of the other cameras, and stores it in the RAM 202.

[0110] The CPU 201 reads the inferred similarity from the RAM 202 and determines whether there is a main subject with a similarity of 50 or more in the image captured by the other camera. If the CPU 201 determines that there is a main subject with a similarity of 50 or more, it determines the positional relationship of the subjects on the input image input to the inference unit 205 based on the subject position information. Specifically, it determines whether the subject with a similarity of 50 or more (the subject in the image captured by the other camera) is to the right or left of the main subject in the input image, based on the magnitude of the x-coordinate of the subject's position.

[0111] When CPU201 has finished determining the positional relationship, or when it has determined that there is no main subject with a similarity threshold of 50 or higher, it increments the camera number by one and performs S1807 with the image captured by the camera 100 (the camera number incremented by one) as the new input image. Then it repeats S1807 until it has finished determining all the captured images.

[0112] Once the CPU 201 has finished judging all the captured images, it determines the positional relationship of camera 100 based on the positional relationship of the subjects in the judged captured images. For example, if a subject that is to the right of the main subject in an image captured by one camera is being captured by another camera as the main subject, the CPU 201 will determine that the other camera is capturing the area to the right of the first camera.

[0113] If the CPU 201 determines that it has been able to determine the positional relationships of all registered cameras, it stores the positional relationships of each camera 100 in the RAM 202 and proceeds to S1809. In other words, it proceeds to S1809 because it was determined that the positional relationships of the subjects could be determined. On the other hand, if it determines that it was not possible to determine the positional relationships of all registered cameras, it proceeds to S1808. In other words, it proceeds to S1808 because it was determined that the positional relationships of the subjects could not be determined.

[0114] Figure 22 shows an example of the relative positions of camera A and camera B. In the image captured by camera A, subject B is visible to the right of main subject A. In other words, when the image captured by camera A is used as the input image, CPU 201 determines that main subject B, with a similarity threshold of 50 or higher, is present in the image captured by camera B. Then, it determines that subject B is visible to the right of main subject A in the input image based on the magnitude of the x-coordinates of the subject positions. Therefore, it is determined that camera B is located to the right of camera A.

[0115] In S1808, the CPU 201 reads the PTZ control value previously stored in the ROM 203, sends a control command based on that control value to the camera 100 via the communication unit 204, and then proceeds to S1801. Specifically, by sending a control value to zoom out, the camera can capture a wider area, or by sending a control value to move the pan or tilt to a different position than the current one, the field of view can be changed. In other words, the communication unit 204 functions as a means of sending control commands.

[0116] Then, steps S1801-S1805 and S1807 are executed again, and if the relative positions of the subjects can be determined, the process proceeds to S1809. On the other hand, if the process proceeds to S1808 even after changing the control values, the control values ​​are changed again and the processing from S1801 is executed. In other words, if the CPU 201 cannot obtain the relative positions of the subjects, the transmission means sends control commands until the relative positions of the subjects can be obtained.

[0117] In this embodiment, the PTZ control value is pre-stored in ROM203, but this is not limited to this. For example, a value entered by the user via the operation I / F206 may be used as the PTZ control value, or the CPU201 may calculate the control value based on the captured image stored in S1801. Furthermore, it is not necessary to transmit all three PTZ control values ​​(zoom out, pan, and tilt); only one of the control values ​​may be transmitted. In addition, the PTZ control value (camera imaging range) may be reset to its initial value (before execution of the flowchart in Figure 18) once the overhead camera is detected and the subject position and camera number have been linked.

[0118] If S1809 is a transition from S1806, it performs the same operation as S507 in Figure 5. If it is a transition from S1807, the CPU 201 reads the positional relationships of each camera stored in S1807 from the RAM 202 and reassigns the camera numbers sequentially from 1 based on those positional relationships.

[0119] As explained above, according to this embodiment, even if a camera equivalent to an overhead camera is not prepared in advance, an existing camera can be made to function as an overhead camera and the subject position can be linked to the camera number.

[0120] <Other Embodiments> Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.

[0121] For example, assignment necessity information indicating whether or not an identification code needs to be assigned may be pre-set in an assignment setting means such as a ROM 203, and the CPU 201 may perform the assignment of the identification code as shown in Figure 5, etc., if the assignment necessity information indicates that an identification code needs to be assigned.

[0122] Furthermore, the present invention also includes cases where a software program that realizes the functions of the above-described embodiments is supplied directly from a recording medium or via wired / wireless communication to a system or device having a computer capable of executing the program, and the program is executed. Therefore, the program code itself supplied to and installed on a computer in order to realize the functional processing of the present invention also realizes the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

[0123] In that case, the form of the program is irrelevant, as long as it possesses the functionality of a program, including object code, programs executed by an interpreter, and script data supplied to the OS.

[0124] The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory. Furthermore, as a method for supplying the program, one possible approach is to store the computer program forming the present invention on a server on a computer network, and then have connected client computers download and run the computer program.

[0125] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist. Embodiments of this disclosure include the following configurations, methods, and programs.

[0126] [Configuration 1] Image acquisition means for acquiring captured images from multiple imaging devices, A subject position acquisition means that acquires the positional relationship of multiple subjects from the captured image acquired by the image acquisition means, An assignment means assigns identification codes to select the plurality of imaging devices in the same order as the arrangement of the plurality of subjects, based on the positional relationship acquired by the subject position acquisition means and the captured image, A selection means that selects one of the plurality of imaging devices based on the identification code assigned by the assignment means, A control device characterized by having the following features. [Configuration 2] The aforementioned plurality of imaging devices include an overhead imaging device that captures an overhead image, The subject position acquisition means acquires the positional relationship using the overhead image captured by the overhead imaging device and the other captured images. The control device according to configuration 1, characterized by the above. [Configuration 3] The system further includes a storage means in which reference images for acquiring the aforementioned positional relationship are stored in advance. The subject position acquisition means acquires the positional relationship using the reference image and the captured image acquired by the image acquisition means. A control device according to configuration 1 or 2, characterized by the above. [Structure 4] The control device according to configuration 1 or 2, characterized in that the subject position acquisition means uses the image with the largest number of subjects among the images acquired by the image acquisition means as a reference image, and acquires the positional relationship using the reference image and the other images. [Composition 5] The control device according to configuration 1 or 2, characterized in that the subject position acquisition means uses the image acquired by the image acquisition means as a reference image, which has the largest number of subjects and the smallest average size of subjects, and acquires the positional relationship using the reference image and the other images. [Composition 6] A main subject determination means determines the main subject of the captured image acquired by the image acquisition means, A classification means for grouping imaging devices that have the same main subject as determined by the main subject determination means into the same group, Priority setting means for setting priority according to the size of the subject in the captured image captured by the imaging device, which has been grouped into the same group by the classification means, It further possesses, The assignment means assigns the identification code based on the positional relationship acquired by the subject position acquisition means, the grouping results by the classification means, and the priority set by the priority setting means. A control device according to any one of configurations 1 to 5, characterized by the above. [Composition 7] The system further includes correspondence information acquisition means for acquiring correspondence information indicating the correspondence between the imaging device and peripheral devices attached to the imaging device, The classification means groups the imaging device and the corresponding peripheral equipment as the same group based on the correspondence information. The control device according to configuration 6, characterized by the above. [Structure 8] The system further includes movement determination means for determining the movement of the subject, If the movement determination means determines that the subject has moved, the subject position acquisition means acquires the positional relationship again, and if the assignment means determines that the positional relationship has changed before and after the reacquisition, it reassigns the identification code. A control device according to any one of configurations 1 to 7, characterized by the above. [Composition 9] A notification means that notifies the user when the movement determination means determines that the subject has moved, Operation information acquisition means for acquiring operation information based on the user's actions, It further possesses, The assignment means assigns the identification code based on the operation information acquired by the operation information acquisition means after notification by the notification means. The control device according to configuration 8, characterized by the above. [Configuration 10] Multiple imaging devices include variable imaging devices that can change the imaging range. The device further includes a transmission means for transmitting a control command to the variable imaging device for changing the imaging range. If the subject position acquisition means is unable to acquire the positional relationship, the transmission means will transmit the control command until the positional relationship can be acquired. A control device according to any one of configurations 1 to 9, characterized by the above. [Composition 11] The system further includes an assignment setting means in which assignment necessity information indicating whether or not the aforementioned identification code needs to be assigned is pre-set, The assignment means performs the assignment of the identification code when the assignment requirement information set in the setting means indicates that the identification code needs to be assigned. The control device according to claim 1. [Method 1] A control method performed by a control device of an imaging device, Image acquisition process, which involves acquiring images from multiple imaging devices, A subject position acquisition step which acquires the positional relationship of multiple subjects from the captured image acquired in the image acquisition step, An assignment step in which, based on the positional relationship obtained in the subject position acquisition step and the captured image, an identification code is assigned to select the plurality of imaging devices in the same order as the arrangement of the plurality of subjects, A control method characterized by having the following features. [program] A program characterized by causing a computer to execute the control method described in configuration 12. [Explanation of Symbols]

[0127] 100 Cameras 200 controllers 201 CPU 202 RAM 203 ROM 204 Communications Department 205 Reasoning section 206 Operation I / F 207 Display section 208 Internal Bus 300 Networks 401 Camera selection button 402 Camera group switching button 403 Zoom Seesaw 404 Joystick

Claims

1. Image acquisition means for acquiring captured images from multiple imaging devices, A subject position acquisition means that acquires the positional relationship of multiple subjects from the captured image acquired by the image acquisition means, An assignment means assigns identification codes to select the plurality of imaging devices in the same order as the arrangement of the plurality of subjects, based on the positional relationship acquired by the subject position acquisition means and the captured image, A selection means that selects one of the plurality of imaging devices based on the identification code assigned by the assignment means, A control device characterized by having the following features.

2. The aforementioned plurality of imaging devices include an overhead imaging device that captures an overhead image, The subject position acquisition means acquires the positional relationship using the overhead image captured by the overhead imaging device and the other captured images. The control device according to feature 1.

3. The system further includes a storage means in which reference images for acquiring the aforementioned positional relationship are stored in advance. The subject position acquisition means acquires the positional relationship using the reference image and the captured image acquired by the image acquisition means. The control device according to feature 1.

4. The control device according to claim 1, wherein the subject position acquisition means uses the image with the largest number of subjects among the images acquired by the image acquisition means as a reference image, and acquires the positional relationship using the reference image and the other images.

5. The control device according to claim 1, wherein the subject position acquisition means uses the image acquired by the image acquisition means as a reference image, which has the largest number of subjects and the smallest average size of subjects, and acquires the positional relationship using the reference image and the other images.

6. A main subject determination means determines the main subject of the captured image acquired by the image acquisition means, A classification means for grouping imaging devices that have the same main subject as determined by the main subject determination means into the same group, Priority setting means for setting priority according to the size of the subject in the captured image captured by the imaging device, which has been grouped into the same group by the classification means, It further possesses, The assignment means assigns the identification code based on the positional relationship acquired by the subject position acquisition means, the grouping results by the classification means, and the priority set by the priority setting means. The control device according to feature 1.

7. The system further includes correspondence information acquisition means for acquiring correspondence information indicating the correspondence between the imaging device and peripheral devices attached to the imaging device, The classification means groups the imaging device and the corresponding peripheral equipment as the same group based on the correspondence information. The control device according to claim 6.

8. The system further includes movement determination means for determining the movement of the subject, If the movement determination means determines that the subject has moved, the subject position acquisition means acquires the positional relationship again, and if the assignment means determines that the positional relationship has changed before and after the reacquisition, it reassigns the identification code. The control device according to feature 1.

9. A notification means that notifies the user when the movement determination means determines that the subject has moved, Operation information acquisition means for acquiring operation information based on the user's actions, It further possesses, The assignment means assigns the identification code based on the operation information acquired by the operation information acquisition means after notification by the notification means. The control device according to claim 8.

10. Multiple imaging devices include variable imaging devices that can change the imaging range. The device further includes a transmission means for transmitting a control command to the variable imaging device for changing the imaging range. If the subject position acquisition means is unable to acquire the positional relationship, the transmission means will transmit the control command until the positional relationship can be acquired. The control device according to feature 1.

11. The system further includes an assignment setting means in which assignment necessity information indicating whether or not the aforementioned identification code needs to be assigned is pre-set, The assignment means performs the assignment of the identification code when the assignment requirement information set in the setting means indicates that the identification code needs to be assigned. The control device according to feature 1.

12. A control method performed by a control device, Image acquisition process, which involves acquiring images from multiple imaging devices, A subject position acquisition step which acquires the positional relationship of multiple subjects from the captured image acquired in the image acquisition step, An assignment step in which, based on the positional relationship obtained in the subject position acquisition step and the captured image, an identification code is assigned to select the plurality of imaging devices in the same order as the arrangement of the plurality of subjects, A control method characterized by having the following features.

13. A program characterized by causing a computer to execute the control method described in claim 12.

Images