Information processing device, information processing method, and program

Abstract

Set the field of view to one that is suitable for tracking multiple subjects. [Solution] The positions of two or more subjects are identified. Based on the positions of the two or more subjects, if their positions satisfy predetermined conditions, the imaging device is controlled by a first control function that images all of the two or more subjects. If their positions do not satisfy the predetermined conditions, the imaging device is controlled by a second control function that performs imaging during a first period in which at least the first of the two or more subjects is imaged, and imaging during a second period in which at least the second of the two or more subjects, which is different from the first subject, is imaged.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, network cameras having an automatic tracking function that performs pan-tilt-zoom control (hereinafter, PTZ control) so as to continuously capture a subject within a screen have become widespread. In automatic tracking, usually, imaging is often performed after designating a single subject as the tracking target, but depending on the use scene, multiple subjects may be set as the tracking targets. For example, when it is desired to capture the main dancers in a group at a dance event or multiple children at a sports meet in a video by tracking them, it is conceivable to perform tracking of multiple subjects. In that case, PTZ control is required so that the designated multiple subjects can be continuously imaged.

[0003] In the technique described in Patent Document 1, the positions of a plurality of subjects designated by the user within the screen are identified by video analysis, and the center of gravity of the plurality of subjects is calculated after taking into account the moving direction and moving speed. Then, by continuously changing the shooting direction so that the center of gravity comes to the center of the screen, the plurality of subjects are included in the angle of view.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Document 1 uses video analysis, which allows for stable positioning of subjects within the frame in scenes where their location is easily identifiable. However, when the distance between subjects increases and simultaneous imaging with the same frame becomes impossible, video analysis cannot identify the location of people outside the frame, making it impossible to set an appropriate frame. Furthermore, once the camera is directed towards a specific person, it becomes difficult to direct the camera towards another person who has moved outside the frame, limiting the applicable use cases. [Means for solving the problem]

[0006] An information processing apparatus according to one embodiment of the present disclosure has the following configuration: that is, it includes a identifying means for identifying the positions of two or more subjects, and a control means for controlling an imaging device based on the positions of the two or more subjects, by a first control content that images all of the two or more subjects if the positions satisfy predetermined conditions, and by a second control content that images at least the first subject among the two or more subjects during a first period, and images at least the second subject, which is different from the first subject, during a second period. [Effects of the Invention]

[0007] Set the field of view to one that is suitable for tracking multiple subjects. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of a system configuration including an information processing device. [Figure 2] A block diagram showing an example of the hardware and functional configuration of an information processing device. [Figure 3] A diagram illustrating direction detection using wireless communication. [Figure 4] A flowchart showing an example of information processing according to Embodiment 1. [Figure 5] A flowchart illustrating an example of a cyclical processing procedure. [Figure 6] A diagram illustrating the patrol information related to Embodiment 1. [Figure 7] A diagram showing an example of a UI for setting patrol information. [Figure 8] A diagram illustrating the information processing according to Embodiment 1. [Figure 9] A flowchart showing an example of information processing according to Embodiment 2. [Figure 10] A diagram illustrating the information processing according to Embodiment 2. [Figure 11] A diagram illustrating the patrol information according to Embodiment 3. [Figure 12] A diagram illustrating the information processing according to Embodiment 3. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0010] (Embodiment 1) Referring to Figure 1, an example of the system configuration of a network system 100 (hereinafter referred to as "system 100") including an information processing device having an imaging function according to Embodiment 1 will be described. System 100 includes an information processing device (network camera) 101, a client device 102, and a wireless device 103.

[0011] In system 100, the information processing device 101 and the client device 102 are connected via a network 104 to enable them to send and receive data to and from each other. The type of network 104 is not particularly limited. For example, network 104 may be implemented as a wired or wireless network. Alternatively, network 104 may be implemented by multiple networks connected via other communication devices.

[0012] Furthermore, the information processing device 101 and the wireless device 103 are connected to each other via a wireless network 105, enabling them to send and receive data to and from one another. The network 105 may be implemented, for example, as a network compliant with the Ultra Wide Band (UWB) specification, which is resistant to radio interference using pulse waves. In this case, the wireless device 103 and the information processing device 101 will be equipped with wireless communication modules compliant with the UWB specification. The type of network 105 is not particularly limited, as long as it is possible to send and receive information between the information processing device 101 and the wireless device 103 via a wireless transmission path. For example, a network compliant with a wireless communication standard such as Bluetooth® 5.1 specification may be applied as network 105.

[0013] The information processing device 101 is an information processing device that performs the processes described below in this embodiment. The information processing device 101 according to this embodiment also has an imaging function and can distribute images (e.g., still images or moving images) corresponding to the imaging results to a device connected via a network (e.g., a client device 102). The information processing device 101 may also have at least some of its operations controlled based on instructions from a device connected via a network (e.g., a client device 102). In the following description, the information processing device 101 will be described as being built into an imaging device (network camera) and controlling the imaging device, but it may also control an imaging device that is implemented as a separate device.

[0014] The client device 102 acquires various data from the information processing device 101 via the network 104 and outputs the data to a predetermined output destination. Thereby, for example, the client device 102 can present an image corresponding to the imaging result by the information processing device 101 to the user via a display device such as a display. Further, the client device 102 may control the operation of the information processing device 101 via the network 104. Note that the UI (User Interface) related to the output of the data transmitted from the information processing device 101 and the control of the operation of the information processing device 101 exemplified above is not particularly limited. For example, a so-called browser may be used as the UI.

[0015] The wireless device 103 is a terminal device held by a subject 106 that is an object of imaging (here, an object of automatic tracking) by the information processing device 101. Although details of the wireless device 103 will be described later, the wireless device 103 is associated with each subject and is used by the information processing device 101 to identify the position of the subject 106. In the present embodiment, although various explanations will be made assuming that the subject 106 is a person, the object to which the subject 106 is applied is not limited to only a person. For example, the subject 106 may be a living thing such as an animal, or a moving body such as an automobile or a robot.

[0016] Referring to FIG. 2, an example of the configuration of the information processing device 101 according to the present embodiment will be described. In FIG. 2, an example of the hardware configuration of the information processing device 101 and an example of the functional configuration that executes each function executed by the hardware configuration are shown. Here, as an example of the functional configuration unit of the information processing device 101, a position specifying unit 213, a wireless tracking unit 214, a mode switching unit 215, a patrol control unit 216, a tracking setting unit 217, and a tracking control unit 218 are shown.

[0017] Furthermore, some of the components shown in Figure 2 can be realized by the CPU, which acts as a computer included in the information processing device 101, executing a computer program stored in the storage unit, which acts as a recording medium. The configuration shown in Figure 2 is just one example, and the system is not limited to this configuration as long as the processes described below can be executed. For example, some of the processes described as being performed by the information processing device 101 may be executed by a device different from the information processing device 101, or by different hardware of the information processing device 101. Such hardware could include, for example, an ASIC or a processor (reconfigurable processor or DSP). Also, as mentioned above, in this embodiment the information processing device 101 is described as having an imaging function, but the imaging device may be implemented as a separate device from the information processing device 101.

[0018] The information processing device 101 may include an imaging unit 201 as an imaging means, a lens control unit 202, and a pan / tilt head 203. The information processing device 101 may also include an A / D conversion unit 204, a camera signal processing unit 205, a storage unit 206, a bus 207, an image analysis unit 208, a compression / decompression unit 209, a wired communication processing unit 210, a wireless communication processing unit 211, and a CPU 212. The components of the information processing device 101 described above are connected to each other via the bus 207 so that data can be sent and received from each other.

[0019] The imaging unit 201 includes a zoom lens, a focus lens, an aperture, and an image sensor such as a CMOS image sensor. The lens control unit 202 performs zoom control and focus control by moving the zoom lens and focus lens along the optical axis. The lens control unit 202 may also control the operation of the aperture.

[0020] In the imaging unit 201, light transmitted through the zoom lens, focus lens, and aperture is guided to the image sensor, where it is photoelectrically converted to generate an electrical analog image signal. The imaging unit 201 then applies sampling amplification to the analog image signal (image signal of the subject image) generated by the image sensor, and outputs the analog image signal to the A / D conversion unit 204.

[0021] The tripod head 203 has a pan drive unit and a tilt drive unit. The CPU 212 controls the tripod head 203 via an actuator (not shown) to realize pan and tilt drives that rotate the imaging unit 201 in the horizontal and vertical directions. This makes it possible to control the shooting direction of the imaging unit 201. The content of the control of the attitude of the imaging unit 201 is not particularly limited, and for example, only pan drive or tilt drive may be performed. In that case, the configuration of the tripod head 203 can be appropriately set according to the direction in which the shooting direction of the imaging unit 201 is controlled, and for example, it may have only one of the pan drive unit and the tilt drive unit.

[0022] The A / D conversion unit 204 converts the analog image signal into a digital image signal and outputs the digital image signal to the camera signal processing unit 205. The camera signal processing unit 205 performs various image processing on the digital image signal converted by the A / D conversion unit 204. Examples of image processing performed by the camera signal processing unit 205 on the digital image signal include offset processing, gamma correction processing, gain processing, RGB interpolation processing, noise reduction processing, and color correction processing. Hereafter, for convenience, the digital image signal after image processing by the camera signal processing unit 205 will be referred to as the video signal to distinguish it from the digital image signal before the image processing.

[0023] The memory unit 206 is implemented by RAM (Random Access Memory), ROM (Read Only Memory), and auxiliary storage devices. RAM is a volatile memory, and for example, SRAM or DRAM may be used as RAM. ROM is a non-volatile memory, and for example, EEPROM or flash memory may be used as ROM. Auxiliary storage devices may be implemented by HDD (Hard Disk Drive) or SSD (Solid State Drive), for example. Programs for realizing the functions of the information processing device 101, and data used when those programs are executed, are stored in ROM or auxiliary storage devices. Furthermore, the above-mentioned programs and data are appropriately loaded into RAM via the bus 207 under the control of the CPU 212 and executed by the CPU 212. This enables each component of the information processing device 101 to function.

[0024] The video analysis unit 208 analyzes the video signal and performs various detection processes such as human body detection, face detection, and motion detection. The video analysis unit 208 outputs the results of the video analysis to the CPU 212 via the bus 207.

[0025] The compression / decompression unit 209 generates compressed data by applying a compression process to the target data (e.g., a video signal). The compression / decompression unit 209 may also restore the original data (e.g., a video signal) by applying a decompression process to the compressed data.

[0026] The wired communication processing unit 210 performs various processes related to realizing communication with an external device (e.g., client device 102) via a wired network (e.g., network 104). The configuration of the wired communication processing unit 210 in which it performs communication processing may be appropriately changed depending on the type of network to which it is connected, or the applicable communication method or communication standard.

[0027] The wireless communication processing unit 211 performs various processes related to realizing communication with an external device (e.g., wireless device 103) via a wireless network (e.g., network 105). The configuration of the wireless communication processing unit 211 in which communication processing is performed may be appropriately changed depending on the type of network to be connected, or the applicable communication method or communication standard. In this embodiment, for convenience, the wireless communication processing unit 211 is assumed to be implemented by a wireless communication module compliant with the UWB specification, but is not limited to this. For example, the wireless communication processing unit 211 may perform communication using other communication standards such as Bluetooth®, not just UWB, in which case the wireless communication processing unit 211 only needs to be equipped with a module capable of processing the protocol defined in the target communication standard.

[0028] The CPU 212, acting as a computer, functions as a control means that controls the operation of each part of the entire device, including the information processing device 101, based on computer programs stored in the storage unit 206, which acts as a storage medium. In the example shown in Figure 2, the CPU 212 includes a location identification unit 213, a wireless tracking unit 214, a mode switching unit 215, a patrol control unit 216, a tracking setting unit 217, and a tracking control unit 218.

[0029] The position identification unit 213 identifies the locations of two or more subjects. In this embodiment, the position identification unit 213 can identify the relative position of the wireless device 103 with respect to the information processing device 101 as the location of the subject, based on the radio waves received from the wireless device 103. In particular, the position identification unit 213 can identify the relative direction of the wireless device 103 as seen from the information processing device 101 based on the direction of arrival of the wireless communication radio waves (radio wave direction), and can also identify the distance between the information processing device 101 and the wireless device 103 based on the radio waves.

[0030] In this embodiment, the positioning unit 213 identifies the position of the subject using the direction detection function and distance measurement function of wireless communication using UWB. However, the configuration is not limited to this as long as the position of the subject can be identified. For example, the positioning unit 213 may identify the position of the subject using the direction detection function and distance measurement function of wireless communication compliant with the Bluetooth® 5.1 specification. Details of the direction detection function and distance measurement function will be described separately with reference to Figure 3.

[0031] The wireless tracking unit 214 performs PTZ control using the lens control unit 202 and the tripod head 203 based on the position of the subject 106 identified by the position identification unit 213. As a result, the wireless tracking unit 214 directs the shooting direction to the position of the subject 106 and adjusts the zoom value according to a predetermined distance, thereby enabling automatic tracking of the subject 106. The tracking process for a predetermined subject can be performed by arbitrarily employing commonly used tracking techniques.

[0032] When identifying the position of a subject using video analysis, it can be difficult to pinpoint the location of each subject in scenes where video analysis is ineffective, such as scenes with many overlapping people or scenes with significant lighting fluctuations. This can also reduce the accuracy of tracking multiple people using the center of gravity of such positions. Furthermore, depending on the use case, the distance between subjects may be large, making it difficult to capture all of the user-specified subjects within the frame. In such cases, even though the user would ideally like to capture all the specified subjects, they may be forced to limit the subjects to be tracked from among multiple people. This is because video analysis cannot pinpoint the position of people outside the frame, and once the camera is directed towards a specific person, it becomes difficult to direct the camera towards another person who has moved out of the frame. Therefore, the use cases in which multi-person tracking using video analysis can be applied are limited. On the other hand, by identifying the position of subjects via wireless communication, tracking accuracy can be maintained even in cases where the accuracy of subject positioning decreases with video analysis.

[0033] The mode switching unit 215 switches the control mode of the imaging device between a first control content and a second control content depending on whether the positions of two or more subjects identified by the position identification unit 213 meet predetermined conditions. The mode switching unit 215 according to this embodiment can switch between a mode in which imaging of subjects is performed with a fixed field of view (fixed imaging mode) and a mode in which imaging is performed with multiple fields of view at intervals (circulating imaging mode) based on predetermined conditions (switching rules). The switching rules used here will be described later with reference to Figure 7.

[0034] The patrol control unit 216 controls the imaging device in patrol imaging mode. Here, the patrol control unit 216 controls the orientation of the imaging device performing imaging so that imaging is performed sequentially at multiple fields of view. The processing by the patrol control unit 216 will be described later with reference to Figure 5.

[0035] The tracking setting unit 217 sets setting information related to the tracking process. In this embodiment, the setting information includes the target to be imaged when tracking a subject, or the switching rules, and a detailed explanation of these will be given later with reference to Figure 7.

[0036] The tracking control unit 218 controls the overall process related to automatic tracking. For example, the tracking control unit 218 can control the start and end of the automatic tracking process, as well as perform initial settings.

[0037] Referring to Figure 3, the principle of how the positioning unit 213 in this embodiment detects the direction of radio waves from the wireless device 103 will be outlined. As a method for detecting the direction of radio waves, for example, a method called AoA (Angle of Arrival) or a method called AoD (Angle of Departure) can be employed. AoA is a method in which the receiving device calculates the reception angle of the radio waves to detect the direction of arrival of the radio waves. AoD is a method in which the transmitting device calculates the radiation angle and then transmits the radiation angle to the receiving device to detect the direction of arrival of the radio waves on the receiving device side. In this embodiment, for convenience, the wireless device 103 is set as the transmitting device and the wireless communication processing unit 211 is set as the receiving device, and the positioning unit 213 detects the direction of radio waves by AoA, but the detection of the direction of radio waves by AoD may also be performed.

[0038] In this embodiment, the positioning unit 213 can determine the direction of radio waves using, for example, the receiver 301 and antenna 302 shown in Figure 3. Here, the positioning unit 213 can use an antenna array having multiple antennas 302 to detect the direction of radio waves based on the phase difference that occurs between the radio wave reception results at each antenna 302, which is due to each antenna 302 being positioned at a different location.

[0039] As shown in Figure 3, the receiving device, the wireless communication processing unit 211, has a receiver 301 and a plurality of antennas 302 arranged at equal intervals. The transmitting device, the wireless device 103, has a transmitter 303 and an antenna 304. In the example shown in Figure 3, for the sake of simplicity, the transmitting device, the wireless device 103, has only one antenna 304, but the number of antennas is not particularly limited. On the transmitting device, the wireless device 103, the transmitter 303 transmits radio waves 305 containing a predetermined direction detection signal from the antenna 304. On the receiving device, the wireless communication processing unit 211, the radio waves 305 transmitted from the wireless device 103 are received by the plurality of antennas 302. At this time, each of the plurality of antennas 302 arranged at equal intervals receives the radio waves 305, but because the distance between antenna 304 and each antenna 302 (for example, distances D1 and D2) is different, each antenna 302 will detect a signal with a different phase. Under these premises, the positioning unit 213 acquires a modulated signal (IQ modulated signal) having in-phase and quadrature phase components as phase information of the radio wave, while switching the active antenna in the wireless communication processing unit 211. Then, the positioning unit 213 calculates the incident angle θ, which is the relative direction of the signal, based on the acquired IQ modulated signal. The calculation of the incident angle θ is performed using information such as the wavelength of the radio wave, the distance between the antennas, and the phase difference, but since the method is publicly known, a detailed explanation is omitted.

[0040] By the method described above, the position identification unit 213 according to this embodiment can determine the relative direction of the subject to the information processing device 101 based on the result of determining the direction of radio waves from the wireless device 103.

[0041] Furthermore, the positioning unit 213 according to this embodiment also measures the distance from the imaging device to the subject 106. The distance measurement method is assumed to be UWB, which calculates distance based on the principle of ToF (Time of Flight) as the communication protocol handled by the positioning unit 213, but it is not limited to such a method as long as the distance to the subject can be similarly calculated or estimated. For example, the positioning unit 213 may use the radio wave strength based on the Bluetooth® specification to calculate the distance to the wireless device 103 as the distance to the subject. The specific distance calculation methods are all publicly known, so we will omit their explanation.

[0042] By the method described above, the positioning unit 213 of the information processing device 101 can determine the distance to the subject relative to the information processing device 101 based on the distance measurement results from the radio waves received from the wireless device 103.

[0043] Next, referring to Figure 4, an example of the processing performed by the information processing device 101 according to this embodiment will be explained, with particular attention to the overall processing that controls the imaging device according to the position of the subject. Figure 4 is a flowchart of an example of the processing performed by the information processing device 101. The processing shown in Figure 4 is started, for example, when the user performs the operation start processing in the client device 102.

[0044] In S401, the tracking control unit 218 performs initial settings related to the operation of the automatic tracking process. Here, the tracking control unit 218 reflects pre-set information regarding automatic tracking, or user setting information stored via the tracking setting unit 217, as initial settings.

[0045] Here, using Figure 7, an example of a settings screen for setting user settings information will be explained. Screen 701 is an example of a settings screen when tracking multiple people, and the tracking setting unit 217 reflects the input content on screen 701 as settings in the system. The tracking setting unit 217 can set the subject to be tracked (for example, specifying the tag ID of the wireless device to be tracked), or set the priority of subjects in tracking, as setting information. The tracking setting unit 217 can also set whether or not to use the patrol imaging mode based on user input (selection via screen 701). When the patrol imaging mode is available, the switching rule can be set in the field for setting the mode switching condition 702, which allows for detailed settings. In this embodiment, the mode switching condition is met when not all subjects are within one (same) field of view, and the switching rule is set so that patrolling outside the current screen is performed. That is, the mode switching unit 215 determines whether or not all subjects are within the current field of view, and decides whether or not to switch modes according to the result of this determination.

[0046] Here, "all subjects fitting within the same field of view" means that the positions of all subjects are within the imaging range at the current zoom magnification of the imaging device. However, this may also mean that the positions of all subjects are within the imaging range at the lower limit of the imaging device's zoom magnification, or within the imaging range at a predetermined range of zoom magnifications set in advance. Thus, "all subjects fitting within the same field of view" means that the subjects are within the same field of view under the imaging conditions desired by the user.

[0047] In S402, the position identification unit 213 identifies the location of the subject. Here, the location of all subjects possessing the wireless device 103 is identified, but for example, the location of only subjects set as tracking targets in the setting information may be identified. In this embodiment, the ID of the wireless device and the subject are pre-associated, and the location of the subject corresponding to the wireless tag ID specified on the UI as shown in Figure 7 may be identified.

[0048] In S403, the mode switching unit 215 determines whether to switch modes, or in this case, whether to perform a patrol process (to perform imaging in patrol imaging mode). In this embodiment, as explained with reference to Figure 7, if all subjects whose positions were identified in S402 fit within the same field of view, the process proceeds to S404 without performing a patrol process; otherwise, the process proceeds to S405 with performing a patrol process.

[0049] In S404, the wireless tracking unit 214 performs imaging using the imaging device so that all subjects are within the field of view, and returns the processing to S402. In this embodiment, it is assumed that all subjects are placed within the field of view while performing control according to predetermined conditions, such as aligning the centroids of the positions of all subjects identified in S402 to the center of the screen. Since various calculation processes in such an imaging method can be performed using known techniques, a detailed explanation is omitted here.

[0050] In S405, the patrol control unit 216 executes the patrol process. Details of the patrol process will be described later with reference to Figure 5.

[0051] In S406, the patrol control unit 216 determines whether to terminate the patrol process. If the patrol process is terminated, the process returns to S402; otherwise, the process returns to S405. Here, similar to the process in S403, the patrol control unit 216 determines whether all subjects fit within the same field of view based on the position of each subject, and if they fit within the same field of view, it can determine to terminate the patrol process. If the patrol process is to continue, this process in S406 may be repeated at predetermined intervals (for example, 5 seconds), or it may be performed when a predetermined operation is performed, and is not particularly limited to a specific execution timing.

[0052] The process shown in Figure 4 involves repeating steps S402 to S406 while imaging is performed by the imaging device, and terminates when the user performs a termination operation (for example, an operation to terminate imaging by the imaging device).

[0053] Referring to Figure 5, an example of the processing of the information processing device 101 according to this embodiment will be explained, with particular attention paid to the processing related to the cyclic control process.

[0054] In S501, the patrol control unit 216 generates patrol information based on the positional relationship of the subjects. The patrol information according to this embodiment is information that sets the field of view in the patrol imaging mode and the subjects corresponding to that field of view (to be imaged at that field of view).

[0055] The following description of the patrol information according to this embodiment will be made with reference to Figure 6. The patrol information 601 is registered to perform two patrols in the patrol imaging mode: "Patrol 1" performed during a first period and "Patrol 2" performed during a second period, with subjects A and B being imaged in Patrol 1 and subject C being imaged in Patrol 2. Furthermore, such patrol information includes the stay time indicating the period during which the patrol is performed (in this case, the time during which imaging is performed at a specified field of view), or the size of the subjects during the patrol. It is also possible to set imaging priority for the patrol process (between subjects or between patrols), but such processing will be described later in Embodiment 3.

[0056] In S502, the patrol control unit 216 determines the next subject to be imaged from among the subjects based on the patrol information. Here, one patrol included in the patrol information is selected, and the subjects set as subjects in the selected patrol are determined to be imaged.

[0057] In S503, the patrol control unit 216 performs imaging so that the subject set as the imaging target fits within the field of view. Here, imaging is performed with the center of gravity of the subject set as the imaging target aligned with the center of the screen. In this patrol imaging mode, the field of view is updated and reflected in real time according to the movement of the subject set as the imaging target.

[0058] In S504, the patrol control unit 216 determines whether to transition (change) the imaging target to the next patrol target. If it determines to transition to the next patrol target, the process returns to S502; otherwise, the process returns to S503 and continues imaging. The conditions for changing the imaging target can be arbitrarily set by the user via a setting screen such as the one shown in Figure 7, but here it is assumed that the imaging target changes from patrol 1 to patrol 2 after a predetermined period of time has elapsed.

[0059] The process shown in Figure 5 involves repeating steps S502 to S504 while imaging is performed by the imaging device, and terminates when the user performs a termination operation (for example, an operation to terminate imaging by the imaging device).

[0060] With this processing, when tracking two or more subjects, the information processing device 101 can control whether to image all subjects with a fixed field of view or to image the subjects while controlling the imaging direction, depending on whether the positions of the subjects meet predetermined conditions. In particular, it becomes possible to control whether to perform imaging in fixed imaging mode or circulating imaging mode depending on whether all subjects fit within the same field of view. Therefore, even if all specified subjects do not fit within the same field of view, it becomes possible to capture all subjects in the image.

[0061] Figures 8(a) to 8(g) are diagrams illustrating specific examples of captured images to explain the processes related to Figures 4 and 5. The image frame 801 shown in Figure 8 schematically represents an image frame, which is the captured image captured by the imaging device. Figures 8(a) to 8(g) will be explained below in relation to the processes in Figures 4 and 5.

[0062] First, referring to Figures 8(a) to 8(c), we will explain the process up to the point when the mode switching unit 215 determines that it is performing a cyclical process.

[0063] Figure 8(a) shows a state where three subjects, subject 802, subject 803, and subject 804, are tracked, and the center of gravity 805 calculated from the positions identified by wireless communication is controlled to be in the center of the screen, so that all subjects are in the same field of view. In the state shown in Figure 8(a), the information processing device 101 determines that all subjects are in the same field of view (No in S403), tracks and images all subjects so that they are in the field of view (S404), and returns to processing in S402.

[0064] Figure 8(b) shows a situation where subject 804 is relatively far from the positions of subjects 802 and 803, making it difficult to fit all subjects into the same field of view. In the situation shown in Figure 8(b), the information processing device 101 determines that not all subjects fit into the field of view (No in S403) and proceeds to processing S405.

[0065] Figure 8(c) shows the state in Figure 8(b) where, based on the relative positions of all subjects, the combination of subjects to be imaged for each round in the round imaging mode is determined. Here, subject 804 is relatively far from the other two subjects relative to the distance between subjects 802 and 803. Therefore, the round control unit 216 sets the round pattern to perform two rounds, as shown by the dotted lines: a round to image subjects 802 and 803, and a round to image subject 804. Here, the round control unit 216 is storing the round information as shown in Figure 6 (in S501).

[0066] In the state shown in Figure 8(c), the patrol control unit 216 sets the subjects to be imaged (corresponding to one field of view) for each patrol and stores them in a form such as shown in Figure 6. Here, the patrol control unit 216 can, for example, calculate the distance from the centroid of the other subjects excluding that subject for each subject, and generate patrol information such that the subject with the longest calculated distance is imaged in a different patrol from the other subjects. Alternatively, for example, the patrol control unit 216 may use a machine learning model that has been trained to generate groups of neighboring subjects based on the position of each subject, and perform imaging for each patrol to generate information on the group of subjects. Furthermore, for example, the patrol control unit 216 may set the subjects to be imaged for each patrol based on user input. In this way, for groups of subjects that do not fit in the same field of view, the method for determining the correspondence is not particularly limited, as long as each subject can be associated with two or more fields of view.

[0067] Next, the control during the patrol process will be explained with reference to Figures 8(d) and 8(e). Figure 8(d) shows the state in which the camera is patrolling with subjects 802 and 803 as tracking targets, and the imaging direction is controlled so that the center of gravity 806, calculated from the positions of subjects 802 and 803, is in the center of the screen. On the other hand, Figure 8(e) shows the state in which the camera is patrolling with subject 804 as the tracking target, and imaging is centered on subject 804. Here, the patrol control unit 216 identifies the patrol destination in S502 and controls the camera to fit the subjects set as patrol targets into the field of view in S503. As a result, by imaging while patrolling the two imaging directions in sequence, it becomes possible to image all subjects.

[0068] Here, during the patrol process, there is a risk that viewers may feel somewhat uneasy. This is because, although it is a patrol, the imaging is temporarily limited to a portion of the designated subjects, which could lead viewers to mistakenly believe that the imaging targets are limited. From this perspective, in the display of the captured images in patrol imaging mode, display control may be performed so that information regarding the control content (in this case, information regarding the patrol) is superimposed. As such information regarding the patrol, for example, the text "Patrol Scheduled" is displayed at a predetermined position on the screen, as shown in displays 807 and 808 in Figures 8(d) and 8(e). The information regarding the patrol according to this embodiment may be information indicating the existence of subjects scheduled for patrol, as shown in displays 807 and 808, or it may be information indicating subjects that are not within the current field of view among the subjects to be imaged, such as "Image Scheduled: Subject 804". Furthermore, the information regarding the patrol is not limited to text information; for example, an image (icon) of the face of a subject scheduled for imaged may be displayed. By performing such processing, it is possible to reduce the anxiety that viewers may feel when performing patrol imaging.

[0069] Finally, referring to Figures 8(f) and 8(g), we will explain the process of transitioning from the circulating imaging mode back to the fixed imaging mode. Figure 8(f) shows the state in which the subject 804 moves in the direction of the arrow from the state shown in Figure 8(c), so that all subjects are once again contained within the same field of view. Here, even in circulating imaging mode, it is assumed that the mode switching unit 215 periodically determines whether or not to switch modes based on the distance between all subjects. In that case, in S403, the mode switching unit 215 determines that all subjects can be contained within the same field of view (no circulating processing is performed), and the circulating control unit 216 terminates the circulating processing in S406 and returns the process to S402.

[0070] Figure 8(g) shows the state shown in Figure 8(f), where subjects 802, 803, and 804 are again being captured in fixed imaging mode with the same field of view. Here, as in the state shown in Figure 8(a), the camera tracks and captures all subjects so that they fit within the field of view.

[0071] With this configuration, when tracking two or more subjects, control can be made to either capture all subjects with a fixed field of view or capture subjects while controlling the imaging direction, depending on whether the positions of the subjects meet predetermined conditions. In particular, it becomes possible to control whether to perform imaging in fixed imaging mode or circulating imaging mode depending on whether all subjects fit within the same field of view. Therefore, it becomes possible to set an appropriate field of view when tracking multiple subjects. In particular, even if all specified subjects do not fit within the same field of view, it becomes possible to capture all subjects in the video.

[0072] [Embodiment 2] In the information processing device 101 according to Embodiment 1, when determining the mode switching as shown in S403, if all subjects fit within the same field of view, the device performs imaging so that all subjects fit within the same field of view and does not perform the circulating process. However, even if subjects fit within the same field of view, if some subjects are separated from other subjects in the depth direction, the visibility of some subjects may decrease. For example, if some subjects are separated from a group in the depth direction, the zoom level will be adjusted to match the group because the group is in the foreground. In that case, the subject in question will appear small, making it difficult to capture details such as facial expressions in the image.

[0073] From this perspective, the information processing device 101 according to Embodiment 2 determines whether there are any subjects that are separated from the group in the depth direction, even when all subjects fit within the same field of view. If there are subjects that are separated in the depth direction, it controls the device to image such subjects at a different field of view than the other subjects. This processing makes it possible to reduce the decrease in visibility by taking into account the spread in the depth direction. The information processing device 101 according to this embodiment has basically the same configuration as that of Embodiment 1 and can perform the same processing, so redundant explanations will be omitted.

[0074] Hereinafter, with reference to Figure 9, an example of the processing performed by the information processing device 101 according to this embodiment will be described, in particular the overall processing that controls the imaging device according to the position of the subject. Figure 9 is a flowchart of an example of the processing performed by the information processing device 101 according to this embodiment. The processing shown in Figure 9 is performed in the same manner as the processing shown in Figure 4, except that S901 is performed between S403 and S404, and S406 is performed instead of S406, so redundant explanations will be omitted.

[0075] In S901, the mode switching unit 215 determines whether there are any subjects that are separated in the depth direction from the group of subjects that fit within the field of view, assuming that all subjects fit within the same field of view (Yes in S403). If it is determined that there are no separated subjects, the process proceeds to S404; otherwise, the process proceeds to S405.

[0076] Here, the mode switching unit 215 can determine, for example, whether there are subjects that are separated from the group of subjects in the depth direction based on the distance from the imaging device to each subject. Here, the distance from the imaging device to the subjects is measured by the position identification unit 213 and such distance is used, but the distance may also be estimated based on an image (for example, by a machine learning model that estimates the distance from the imaging device to each subject using an image as input), and any known technique can be used for calculating the distance from the imaging device to the subjects. Using such distance information, the mode switching unit 215 groups the subjects according to predetermined conditions based on the calculated distance from the imaging device for each subject, and calculates the centroid of the distance for each group. Then, if there are subjects that are separated by a predetermined value (fixed value or percentage) or more compared to the centroid distance in other groups, the mode switching unit 215 can determine that such subjects are separated from the group of subjects in the depth direction. Such grouping can be arbitrarily set according to the required conditions by known statistical processing.

[0077] The mode switching unit 215 may also determine whether or not there are subjects that are separated in the depth direction from the group of subjects, based on the image, without using the distance from the imaging device to the subjects. For example, the mode switching unit 215 can calculate the area of ​​each rectangular region surrounding the subjects, and if there are subjects whose area is significantly larger or smaller than that of other subjects, it can determine that such subjects are separated in the depth direction from the group of subjects. The detection of subjects whose area is significantly larger or smaller than that of other subjects can also be performed by known statistical processing, and a detailed explanation will not be given here. Alternatively, the mode switching unit 215 may also determine whether or not there are subjects that are separated in the depth direction from the group of subjects, using a machine learning model that detects subjects separated in the depth direction from the group of subjects as input to the image.

[0078] In S902, the patrol control unit 216 determines whether or not to terminate the patrol process. Here, in addition to the control conditions for the patrol process as described in S406, the patrol control unit 216 may terminate the patrol process if there are no subjects that are far away in the depth direction from the group of subjects that fit within the field of view. If it is determined in S901 that there are subjects that are far away in the depth direction, the process in S902 may be executed by monitoring the distance from the imaging device to such subjects, or each process may be executed based on the position of the entire group of subjects each time S902 is executed.

[0079] Figures 10(a) to 10(g) are examples of specific captured images used to explain the processing described in Figure 9. Figures 10(a) to 10(g) correspond to Figures 8(a) to 8(g), respectively, except that subject 803 and subject 804 are moving away in the depth direction rather than subject 804 moving out of the field of view. Therefore, redundant explanations are omitted.

[0080] First, referring to Figures 10(a) to 10(c), we will explain the process up to the point when the mode switching unit 215 determines that it is performing a cyclical process. Figure 10(a) shows the same state as Figure 8(a).

[0081] Figure 10(b) shows a state in which subjects 803 and 804 move in the depth direction relative to subject 802, and visibility decreases when all subjects are imaged at the same zoom level. In the state shown in Figure 10(b), the patrol control unit 216 determines that there is a subject that is far away from the group in the depth direction (No in S901) and proceeds to processing in S405.

[0082] Figure 10(c) shows the state in which the combination of subjects to be circulated is determined based on the positional relationship of all subjects in the state shown in Figure 10(b). Here, subjects 803 and 804 are relatively far from subject 802 in the depth direction. Therefore, the circulating control unit 216 sets the circulating pattern to perform two circulating patterns, as shown by the dotted lines: a circulating pattern to image subject 802 and a circulating pattern to image subjects 803 and 804.

[0083] In the state shown in Figure 10(c), the patrol control unit 216 sets the subject to be imaged (corresponding to one field of view) for each patrol and stores it in a form such as shown in Figure 6. Here, the patrol control unit 216 calculates the difference between the distance from the imaging device to each subject and the centroid of the distances to all other subjects excluding that subject, and generates patrol information so that the subject with the largest difference in the calculated distance is imaged in a different patrol than the other subjects. The method for determining the correspondence between subjects and fields of view here is not particularly limited, as long as each subject can be associated with two or more fields of view, as explained in Figure 8(c).

[0084] Next, the control during the patrol process will be explained with reference to Figures 10(d) and 10(e). Figure 10(d) shows the state in which the camera is patrolling with subject 802 as the tracking target, and the control is set so that subject 802 is in the center of the screen. On the other hand, Figure 10(e) shows the state in which the camera is patrolling with subjects 803 and 804 as the tracking targets, and the imaging is performed so that the centers of gravity of subjects 803 and 804 are in the center of the screen.

[0085] Finally, referring to Figures 10(f) and 10(g), the process of transitioning from the circulating imaging mode back to the fixed imaging mode will be explained. Figure 10(f) shows a state where subjects 803 and 804 have moved closer to subject 802 in the depth direction, and it is no longer visually problematic to image them again at the same zoom level. Here, even in circulating imaging mode, it is assumed that the mode switching unit 215 periodically determines whether or not to perform a mode switch. In that case, in S403 and S901, the mode switching unit 215 has determined that all subjects can be contained in the same field of view and that there are no subjects that are far from the group in the depth direction (no circulating processing), and in S902 the circulating control unit 216 terminates the circulating processing and returns to S402.

[0086] Figure 10(g) shows the state shown in Figure 10(f), where subjects 802, 803, and 804 are again being captured in fixed imaging mode with the same field of view. Here, as in the state shown in Figure 10(a), the camera tracks and captures all subjects so that they fit within the field of view.

[0087] With this configuration, when tracking two or more subjects, if there are subjects that are far apart in the depth direction, the system can control the imaging direction while capturing the subjects. Therefore, it is possible to reduce the decrease in visibility by taking into account the depth of the scene.

[0088] [Embodiment 3] In the embodiments 1 and 2 described above, when performing the patrol processing as shown in S503, no priority is set for each patrol process, and it was explained that there is no difference in control during each patrol. However, there are cases where it is desirable to set a priority even among the specified subjects. For example, when trying to capture both one's child and their friend on video, one may want to prioritize capturing one's child for a longer period. From this perspective, the information processing device 101 according to this embodiment sets an imaging priority for each patrol process in the same patrol processing as in embodiment 1, and executes the patrol process according to that priority.

[0089] The following description of the patrol information according to this embodiment will be made with reference to Figure 11. Similar to the patrol information 601 in Figure 6, the patrol information 1101 is registered to perform two patrols, "Patrol 1" and "Patrol 2," in the patrol imaging mode, with subjects A and B being imaged in Patrol 1 and subject C being imaged in Patrol 2. Furthermore, in this case, subject C (corresponding to tag ID "C") is set to have a higher priority than other subjects, and the priority of Patrol 2, which includes subject C, is higher than that of Patrol 1. Thus, the priority in the patrol processing (patrol control) may be set for each subject or for each patrol processing.

[0090] Here, the patrol control unit 216 may set the duration (imaging time) of high-priority patrols to be longer than other patrols, or the display size of the subject to be larger than other patrols (for example, by increasing the zoom magnification during imaging), and can perform arbitrary priority processing. In the patrol information 1101, the stay time, which is the time for imaging, is set to 3 seconds for patrol 1 and 5 seconds for patrol 2, and the display size according to the zoom magnification is set to "small" for patrol 1 and "large" for patrol 2. Such changes in display size may be made by changing the zoom magnification of the imaging device as described above, or by image processing in conjunction with the video analysis unit 208.

[0091] In this explanation, the priority in the rotation control is set according to the priority set for each subject. However, the process is not limited to this, as long as it is possible to set a higher priority through the rotation of priority imaging. For example, a high-priority position may be set, and the rotation of imaging of subjects located at such a position may be set to a higher priority. The position for which this priority is set may be specified by coordinates that can be associated with the position of the subject identified by the position identification unit 213, and a main coordinate range with a higher priority and a sub-coordinate range with a lower priority may be set. Furthermore, such priority settings may be performed on the UI shown in Figure 7.

[0092] Figure 12 is a diagram showing a specific example of captured images to illustrate the priority-based cyclic processing according to this embodiment. The following explanation will omit any parts that overlap with the explanation in Figure 8.

[0093] The control during the patrol process will be explained with reference to Figures 12(a) and 12(b). Figure 12(a) shows an example of an image captured during patrol 1 in the patrol information 1101 shown in Figure 11. Here, it is assumed that subjects 802 and 803 possess wireless devices 103 with IDs A and B, respectively. On the other hand, Figure 12(b) shows an example of an image captured during patrol 2 in the patrol information 1101 shown in Figure 11. Here, it is assumed that subject 804 possesses a wireless device 103 with ID C.

[0094] According to patrol information 1101, in patrol 2, which has a higher priority, subject 804 is imaged with a higher zoom magnification and larger size than subjects 802 and 803. Furthermore, subject 804 is imaged for a longer period than subjects 802 and 803 before the imaging transitions to patrol 1.

[0095] This type of processing allows for prioritizing patrols by setting priorities in the control of the patrol, making it possible to prioritize patrols that meet desired conditions. In particular, it allows for capturing subjects of priority in a way that better suits the user's preferences.

[0096] [Other embodiments] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0097] Furthermore, various modifications may be made as long as they do not deviate from the basic technical concept of the embodiment of this disclosure described above.

[0098] Furthermore, even if a person moves, they may immediately return to their original position, and if the system immediately switches between fixed imaging mode and patrol imaging mode each time, the control may change frequently, potentially reducing visibility. From this perspective, in the mode switching determination process described above as being performed by the mode switching unit 215, a predetermined waiting time may be provided when transitioning to the patrol process. Using Figure 4 as an example, in S403, the mode switching unit 215 may proceed to S405 if the state in which not all subjects fit within the field of view continues for a predetermined time (or longer). Similarly, a waiting time may be provided in the determination of whether or not there are subjects that are far away in the depth direction, as explained with reference to Figure 9.

[0099] Furthermore, in the embodiments described above, an example was given in which so-called PTZ control is applied as the control of the imaging device. However, if it is possible to control the range captured by the imaging device, the imaging device may be controlled by a method other than PTZ control. For example, the same process may be performed even in situations where electronic control such as cropping a part of the imaging device's field of view as the range of the object to be captured is applied to at least part of the control of the shooting direction. Also, for example, the position of the imaging device may be movable.

[0100] The disclosures herein include the following information processing devices, information processing methods, and programs. (Item 1) A means for identifying the locations of two or more subjects, Based on the two or more subject locations mentioned above, When the aforementioned position satisfies predetermined conditions, the imaging device is controlled by a first control configuration that images all of the two or more subjects. If the position does not satisfy the predetermined conditions, the imaging device is controlled by a second control program which includes imaging during a first period in which at least one of the two or more subjects is imaged, and imaging during a second period in which at least one of the two or more subjects, different from the first subject, is imaged. Control means and An information processing device equipped with the following features. (Item 2) The information processing apparatus according to item 1, characterized in that the predetermined condition includes the fact that all of the two or more subjects fit within the first field of view of the imaging device. (Item 3) The information processing device according to item 2, characterized in that all of the two or more subjects fit within the first field of view of the imaging device, and all of the two or more subjects fit within the imaging range at a predetermined zoom magnification range of the imaging device. (Item 4) The information processing apparatus according to item 2 or 3, characterized in that the predetermined condition includes the state in which all of the two or more subjects are contained within the first field of view of the imaging device for a predetermined period of time. (Item 5) The system further includes a determination means for determining whether or not there is a subject among the two or more subjects that is separated from the other subjects in the depth direction, The information processing device according to any one of items 1 to 4, characterized in that the predetermined condition is further determined by the determination means that among the two or more subjects, there is no subject that is separated from the other subjects in the depth direction. (Item 6) The information processing device according to any one of items 1 to 5, further comprising a display control means for displaying an image captured under the second control content with information relating to the second control content superimposed on it. (Item 7) The information processing device according to item 6, characterized in that the information relating to the second control content is information about a subject that is not being imaged in the current field of view. (Item 8) The system further comprises a first setting means for setting a first priority for imaging during the first period and a second priority for imaging during the second period. The information processing apparatus according to any one of items 1 to 7, characterized in that the control means controls imaging in the first period and imaging in the second period according to the first priority and the second priority. (Item 9) The information processing device according to item 8, characterized in that the length of the first period and the second period is set based on the first priority and the second priority. (Item 10) The information processing apparatus according to item 8 or 9, characterized in that the control means performs imaging during a first period with a display size of the subject based on the first priority, and performs imaging during a second period with a display size of the subject based on the second priority. (Item 11) The information processing device according to any one of items 8 to 10, characterized in that the first setting means sets a first priority based on the subject captured in imaging during a first period, and sets a second priority based on the subject captured in imaging during a second period. (Item 12) An information processing device according to any one of items 1 to 11, further comprising a second setting means for setting the imaging range in imaging during the first period based on the position of the subject to be imaged during the first period, and setting the imaging range in imaging during the second period based on the position of the subject to be imaged during the second period. (Item 13) The information processing device according to any one of items 1 to 12, further comprising a third setting means for setting, among the two or more subjects, a subject to be imaged during the first period and a subject to be imaged during the second period. (Item 14) The information processing device according to any one of item 13, wherein the identifying means identifies the location of a device associated with a subject by wireless communication, as the location of each subject. (Item 15) A step of identifying the locations of two or more subjects, Based on the two or more subject locations mentioned above, When the aforementioned position satisfies predetermined conditions, the imaging device is controlled by a first control configuration that images all of the two or more subjects. If the position does not satisfy the predetermined conditions, the imaging device is controlled by a second control program which includes imaging during a first period in which at least one of the two or more subjects is imaged, and imaging during a second period in which at least one of the two or more subjects, different from the first subject, is imaged. The process, An information processing method comprising: (Item 16) A program to cause a computer to function as one of the information processing devices described in any one of items 1 through 14.

[0101] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0102] 101: Information processing device, 102: Client device, 103: Wireless device, 104: Network, 105: Network, 106: Subject

Claims

1. A means for identifying the locations of two or more subjects, Based on the positions of the two or more subjects, When the aforementioned position satisfies predetermined conditions, the imaging device is controlled by a first control configuration that images all of the two or more subjects. If the position does not satisfy the predetermined conditions, the imaging device is controlled by a second control program which includes imaging during a first period in which at least one of the two or more subjects is imaged, and imaging during a second period in which at least one of the two or more subjects, different from the first subject, is imaged. Control means and An information processing device equipped with the following features.

2. The information processing apparatus according to claim 1, characterized in that the predetermined condition includes all of the two or more subjects being contained within the first field of view of the imaging device.

3. The information processing apparatus according to claim 2, characterized in that all of the two or more subjects fit within the first field of view of the imaging device, or all of the two or more subjects fit within the imaging range at a predetermined zoom magnification range of the imaging device.

4. The information processing apparatus according to claim 2, characterized in that the predetermined condition includes the state in which all of the two or more subjects are contained within the first field of view of the imaging device for a predetermined period of time.

5. The system further includes a determination means for determining whether or not there is a subject among the two or more subjects that is separated from the other subjects in the depth direction, The information processing apparatus according to claim 1, further characterized in that the predetermined condition is determined by the determination means to be such that there is no subject among the two or more subjects that is separated from other subjects in the depth direction.

6. The information processing apparatus according to claim 1, further comprising a display control means for displaying an image captured under the second control content with information relating to the second control content superimposed on it.

7. The information processing apparatus according to claim 6, characterized in that the information relating to the second control content is information about a subject that is not being imaged in the current field of view.

8. The system further comprises a first setting means for setting a first priority for imaging during the first period and a second priority for imaging during the second period. The information processing apparatus according to claim 1, characterized in that the control means controls imaging in the first period and imaging in the second period according to the first priority and the second priority.

9. The information processing apparatus according to claim 8, characterized in that the length of the first period and the second period is set based on the first priority and the second priority.

10. The information processing apparatus according to claim 8, characterized in that the control means performs imaging during a first period with a display size of the subject based on the first priority, and performs imaging during a second period with a display size of the subject based on the second priority.

11. The information processing apparatus according to claim 8, characterized in that the first setting means sets a first priority based on the subject captured in imaging during a first period, and sets a second priority based on the subject captured in imaging during a second period.

12. The information processing apparatus according to claim 1, further comprising a second setting means for setting the imaging range in imaging during the first period based on the position of the subject to be imaged during the first period, and setting the imaging range in imaging during the second period based on the position of the subject to be imaged during the second period.

13. The information processing apparatus according to claim 1, further comprising a third setting means for setting, among the two or more subjects, a subject to be imaged during the first period and a subject to be imaged during the second period.

14. The information processing apparatus according to claim 1, characterized in that the identifying means identifies the location of a device associated with a subject by wireless communication, as the location of each subject.

15. A step of identifying the locations of two or more subjects, Based on the positions of the two or more subjects, When the aforementioned position satisfies predetermined conditions, the imaging device is controlled by a first control configuration that images all of the two or more subjects. If the position does not satisfy the predetermined conditions, the imaging device is controlled by a second control program which includes imaging during a first period in which at least one of the two or more subjects is imaged, and imaging during a second period in which at least one of the two or more subjects, different from the first subject, is imaged. The process, An information processing method comprising:

16. A program for causing a computer to function as one of the means of an information processing device according to any one of claims 1 to 14.

Images