Information processing apparatus, information processing method, and program

Abstract

To enable realization of effective voice output.SOLUTION: An information processing apparatus determines whether a target subject is present in a first image acquired by a first imaging apparatus and a second image acquired by a second imaging apparatus, and controls the sound volume of a first speaker corresponding to the first imaging apparatus and a second speaker corresponding to the second imaging apparatus. When determining that the target subject is present in both the first image and the second image, the information processing apparatus performs first control, when determining that the target subject is present in any one of the first image and the second image, performs second control different from the first control, and when determining that the target subject is not present in any of the first image and the second image, performs third control different from the first control and the second control.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing technique used to control an apparatus that captures images and outputs audio. [Background technology]

[0002] Some network cameras are equipped with so-called pan, tilt, and zoom functions. Hereinafter, a network camera will be referred to as a NW camera, pan, tilt, and zoom will be referred to as PTZ, and a camera with PTZ functions will be referred to as a PTZ camera. Some NW cameras are equipped with a wide-angle camera capable of capturing images with a wide angle of view like a panoramic camera, and a speaker capable of outputting audio, in addition to the PTZ camera. With this NW camera, for example, a subject of interest (e.g., a person) captured in an image captured by the wide-angle camera can be tracked by the PTZ camera, and audio can be output from the speaker, allowing the subject of interest (person) to be spoken to (warned, called attention, etc.).

[0003] Patent Document 1 discloses a technology that includes a general first speaker and a directional second speaker, and adjusts the orientation of the second speaker and the volume of each speaker based on the positions of the first person and the second person captured in a camera image. In Patent Document 1, the orientation of the second speaker is adjusted so that the second person is within the range of the directivity of the second speaker and the first person is not within the range. Then, the volume of each speaker is adjusted so that the volume of the first speaker that reaches the first person is within a predetermined range, and the volume of the second speaker that reaches the second person is within a predetermined range. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-197381 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned NW camera capable of audio output, the direction of the audio output from the speaker may deviate from the direction of the target subject tracked by the PTZ camera. In this case, not only is it impossible to effectively call out to the target subject, but the voice of the call may affect other subjects other than the target subject. It is possible to apply the technology of Patent Document 1 to the NW camera, but if the direction of the audio output from the speaker deviates from the direction of the target subject, it is also impossible to effectively call out to the target subject, and the voice of the call may affect other subjects.

[0006] Therefore, an object of the present invention is to enable the realization of effective audio output. [Means for solving the problem]

[0007] The information processing device of the present invention has a determination means for determining whether or not a subject of interest is present in a first image acquired by a first imaging device and a second image acquired by a second imaging device, and a control means for controlling the volume of a first speaker corresponding to the first imaging device and a second speaker corresponding to the second imaging device, wherein the control means performs a first control when the determination means determines that the subject of interest is present in both the first image and the second image, performs a second control different from the first control when the determination means determines that the subject of interest is present in one of the first image and the second image, and performs a third control different from the first control and the second control when the determination means determines that the subject of interest is not present in either the first image or the second image. Effect of the Invention

[0008] According to the present invention, effective audio output can be achieved. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram illustrating an example of a functional configuration of a network camera including an information processing device. [Diagram 2] FIG. 1 is a diagram showing an example of a schematic external configuration of a network camera. [Diagram 3] 4 is a flowchart showing a flow of information processing in the first embodiment. [Figure 4] FIG. 2 is a diagram used to explain the relationship between the shooting direction, the sound output direction, and a target subject. [Diagram 5] 10 is a flowchart showing a flow of information processing in a second embodiment. [Figure 6] 2 is a diagram showing a hardware configuration of a control unit and its peripheral components. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (conditions of use, environment of use, etc.). In each of the following embodiments, the same or similar configurations and processing steps are given the same reference symbols, and duplicated explanations will be omitted.

[0011] First Embodiment Fig. 1 is a block diagram showing an example of the functional configuration of a main part of a network camera (referred to as a NW camera) as an application example of an information processing device according to an embodiment, and Fig. 2 is a diagram showing an example of a schematic external configuration of the NW camera of this embodiment. In the NW camera of this embodiment, the first imaging unit 41 is an imaging device having an imaging optical system with a zoom function and an imaging element. The imaging optical system is composed of a plurality of lenses, an aperture, and an optical filter such as an infrared cut filter. The imaging element is composed of a photoelectric conversion element such as a CCD or a CMOS. CCD is an abbreviation for Charge Coupled Device, and CMOS is an abbreviation for Complementary Metal Oxide Semiconductor.

[0012] The first imaging control unit 11 controls the imaging operation of the first imaging unit 41, the zoom operation of the imaging optical system, the output operation of the captured image, etc. In addition, the first imaging control unit 11 determines the angle of view of the first imaging unit 41 from the zoom amount of the imaging optical system of the first imaging unit 41.

[0013] The first speaker 61 is an audio output device (a device in which a speaker unit is built into a housing) having a relatively narrow directivity, and converts an audio signal input from the first volume control unit 31 into air vibrations. In the case of this embodiment, the first speaker 61 is a speaker that has a maximum sound pressure of 60 dB at a distance of 30 m, for example.

[0014] The first volume control unit 31 adjusts the volume of the sound output from the first speaker 61 under the control of the control unit 1. The first volume control unit 31 is configured to include a D / A converter that converts the sound data to be output to the first speaker 61 into an analog signal, an amplifier for driving the first speaker 61, and a digital volume for adjusting the sound volume, an adjustment circuit for the gain of the amplifier, and the like. In this embodiment, the first volume control unit 31 outputs a sound corresponding to the sound data from the first speaker 61 at a volume corresponding to the volume setting information, based on the sound data and volume setting information supplied from the control unit 1 described later.

[0015] The first imaging unit 41 and the first speaker 61 are mounted on a first camera-to-camera head 71 shown in Fig. 2, and the first camera-to-camera head 71 is a pan-tilt camera-to-camera head that can be driven in the pan and tilt directions, and is driven in the pan and tilt directions by a first camera-to-camera head drive unit 51. The first camera-to-camera head drive unit 51 drives the first camera-to-camera head 71 in the pan and tilt directions, for example, by a motor. As described above, the imaging optical system of the first imaging unit 41 is capable of zooming, and the first camera-to-camera head 71 is a pan-tilt camera-to-camera head, so that the first imaging unit 41 and the first camera-to-camera head 71 form a PTZ camera that can pan, tilt, and zoom.

[0016] The first tripod head control unit 21 drives the first tripod head 71 in the pan direction and the tilt direction by controlling the drive of the first tripod head drive unit 51 under the control of the control unit 1 described later. That is, in this embodiment, the first tripod head control unit 21 functions as a direction control unit that changes and controls the orientation of the first imaging unit 41 (i.e., the shooting direction) and the orientation of the first speaker 61 (audio output direction) in the pan and tilt directions. In addition, the first tripod head control unit 21 transmits the drive amount of the first tripod head 71 by the first tripod head drive unit 51 or the output of an encoder or the like provided on the first tripod head 71 to the control unit 1 described later as information indicating the results of the pan control and tilt control of the first tripod head 71.

[0017] The second imaging unit 42 is an imaging device having an imaging optical system and an imaging element capable of capturing an image with a wider angle of view than the first imaging unit 41. The imaging optical system of the second imaging unit 42 is composed of a plurality of lenses, an aperture, and an optical filter such as an infrared cut filter, and the imaging element is composed of a photoelectric conversion element such as a CCD or a CMOS. The imaging optical system of the second imaging unit 42 has a zoom function, and the angle of view achieved by the zoom function is wider than the angle of view achieved by the imaging optical system of the first imaging unit 41. The second imaging unit 42 may also be a panoramic camera.

[0018] The second imaging control unit 12 controls the imaging operation of the second imaging unit 42, the zoom operation of the imaging optical system, the output operation of the captured image, etc. In addition, the second imaging control unit 12 determines the angle of view at the time of shooting of the second imaging unit 42 from the zoom amount in the imaging optical system of the second imaging unit 42.

[0019] 2, the second camera head 72 is a tilt camera head that can be driven in a tilt direction, and is driven in the tilt direction by the second camera head drive unit 52. The second camera head drive unit 52 drives the second camera head 72 in the tilt direction by, for example, a motor.

[0020] The second camera-head control unit 22 drives the second camera-head 72 in the tilt direction by controlling the drive of the second camera-head drive unit 52 under the control of the control unit 1 described later. That is, in this embodiment, the second camera-head control unit 22 functions as a direction control unit that changes and controls the orientation (shooting direction) of the second imaging unit 42 in the tilt direction. In addition, the second camera-head control unit 22 transmits the amount of drive of the second camera-head 72 by the second camera-head drive unit 52 or the output of an encoder or the like provided on the second camera-head 72 to the control unit 1 described later as information indicating the result of the tilt control of the second camera-head 72.

[0021] The second speaker 62 is an audio output device (a device in which a speaker unit is built into a housing) having a relatively narrow directivity, and converts the audio signal input from the second volume control unit 32 into air vibrations. In the case of this embodiment, the second speaker 62 is a speaker that has a maximum sound pressure of 60 dB at a distance of, for example, 50 m.

[0022] As described above, the first speaker 61 has a maximum sound pressure of 60 dB at a distance of 30 m, whereas the second speaker 62 has a maximum sound pressure of 60 dB at a distance of 50 m. In other words, the first speaker 61 and the second speaker 62 have different distances at which the same maximum sound pressure can be obtained, in other words, different maximum sound pressures at the same distance. Thus, in this embodiment, the second speaker 62 is a speaker that has a longer distance at which the same maximum sound pressure can be obtained than the first speaker 61.

[0023] The second volume control unit 32 adjusts the volume of the sound output from the second speaker 62 under the control of the control unit 1. The second volume control unit 32 is configured to include a D / A converter that converts the sound data to be output to the second speaker 62 into an analog signal, an amplifier for driving the second speaker 62, and a digital volume for adjusting the sound volume, an adjustment circuit for the gain of the amplifier, etc. Based on the sound data and volume setting information supplied from the control unit 1 described later, the second volume control unit 32 outputs sound corresponding to the sound data from the second speaker 62 at a volume corresponding to the volume setting information.

[0024] 2, the third pan head 73 is a tilt pan head that can be driven in a tilt direction, and is driven in the tilt direction by the third pan head drive unit 53. The third pan head drive unit 53 drives the third pan head 73 in the tilt direction by, for example, a motor.

[0025] The third camera-head control unit 23 drives the third camera-head 73 in the tilt direction by controlling the drive of the third camera-head drive unit 53 under the control of the control unit 1 described later. That is, in this embodiment, the third camera-head control unit 23 functions as a direction control unit that changes and controls the orientation (audio output direction) of the second speaker 62 in the tilt direction. In addition, the third camera-head control unit 23 transmits the amount of drive of the third camera-head 73 by the third camera-head drive unit 53 or the output of an encoder or the like provided on the third camera-head 73 to the control unit 1 described later as information indicating the result of the tilt control of the third camera-head 73.

[0026] The second and third camera-to-camera heads 72 and 73 are mounted on a fourth camera-to-camera head 74 shown in Fig. 2, and the fourth camera-to-camera head 74 is a panning head that can be driven in the panning direction, and is driven in the panning direction by a fourth camera-to-camera head drive unit 54. The fourth camera-to-camera head drive unit 54 drives the fourth camera-to-camera head 74 in the panning direction, for example, by a motor. The fourth camera-to-camera head drive unit 54 is driven and controlled by a fourth camera-to-camera head control unit 24, which drives and controls the fourth camera-to-camera head drive unit 54 under the control of a control unit 1 described later. The fourth camera-to-camera head control unit 24 also transmits the amount of drive of the fourth camera-to-camera head 74 by the fourth camera-to-camera head drive unit 54 or the output of an encoder or the like provided on the fourth camera-to-camera head 74 to a control unit 1 described later, as information indicating the result of pan control of the fourth camera-to-camera head 74.

[0027] In this way, since the second camera head 72 is mounted on the fourth camera head 74, the second imaging unit 42 mounted on the second camera head 72 moves in the tilt direction by the second camera head 72 and moves in the pan direction by the fourth camera head. That is, in this embodiment, the second camera head control unit 22 and the fourth camera head control unit 24 controlled by the control unit 1 function as a direction control unit that changes and controls the orientation of the second imaging unit 42 to the tilt and pan directions. Similarly, since the third camera head 73 is mounted on the fourth camera head 74, the second speaker 62 mounted on the third camera head 73 moves in the tilt direction by the third camera head 73 and moves in the pan direction by the fourth camera head. That is, in this embodiment, the third camera head control unit 23 and the fourth camera head control unit 24 controlled by the control unit 1 function as a direction control unit that changes and controls the orientation of the second speaker 62 to the tilt and pan directions. Also, the second imaging unit 42 and the second speaker 62 can be moved in the tilt direction independently by the second camera head 72 or the third camera head 73, whereas the pan direction is moved in the same direction by the fourth camera head 74. Note that the positional relationship between the second imaging unit 42 and the second camera head 72 and the second speaker 62 and the third camera head 73 may be reversed from that in the example of FIG.

[0028] The image processing unit 2 executes predetermined image processing, such as color balance processing and gamma correction processing, on the image data from the first imaging unit 41 and the second imaging unit 42. The image processing unit 2 also generates an image file of a still image or a video in a predetermined format, such as JPEG (Joint Photographic Experts Group), H.264, or H.265, from the image data after the predetermined image processing. The image processing unit 2 then outputs the data of the image file to the memory 3 and the control unit 1.

[0029] The memory 3 has a volatile memory such as SRAM or DRAM, and a non-volatile memory such as a flash memory, and stores data output from the image processing unit 2. SRAM stands for Static Random Access Memory, and DRAM stands for Dynamic Random Access Memory. The memory 3 also stores information for identifying a target subject, such as face information, a person's whole-body image, body type information, clothing information, and the like. The memory 3 also stores information on the directional angle of the first speaker 61 and the directional angle of the second speaker 62, and data on sounds output from the first and second speakers 62, and the like.

[0030] The communication unit 4 is a device for performing wired or wireless communication between the NW camera of this embodiment and an external device (information terminal, etc.). The communication unit 4 transmits to the outside image data (image file) captured by the NW camera of this embodiment and processed by the image processing unit 2. The communication unit 4 also receives control data sent from an external information terminal, information on a target subject (described later), audio data to be output from the first speaker 61 and the second speaker 62, etc.

[0031] The control unit 1 is a control device that controls the entire NW camera of this embodiment. The control unit 1 transmits information related to control to each of the components, namely, the image processing unit 2, the memory 3, the first imaging control unit 11, the first camera head control unit 21, the second imaging control unit 12, the second camera head control unit 22, the second volume control unit 32, the third camera head control unit 23, the fourth camera head control unit 24, and the communication unit 4. The control unit 1 also receives various information and data from each of these components. The main control functions of the control unit 1 will be described below.

[0032] The control unit 1 transmits information such as the start and end of the imaging operation in the first imaging unit 41 and the zoom amount to the first imaging control unit 11 as control information for the first imaging control unit 11. As a result, the first imaging control unit 11 controls the start and end of the imaging operation in the first imaging unit 41, controls the zoom, etc. Furthermore, the control unit 1 receives information on the zoom control result in the first imaging unit 41 from the first imaging control unit 11, and determines the zoom amount, i.e., the angle of view, of the first imaging unit 41 at the current time based on the received zoom control result.

[0033] Moreover, the control unit 1 transmits information such as the start and end of the sound output operation in the first speaker 61, volume setting information, and sound output from the first speaker 61 (sound data corresponding to the content of the call to the target subject) to the first volume control unit 31. In the case of this embodiment, the sound output from the first speaker 61 (sound corresponding to the content of the call) is stored in the memory 3, and the control unit 1 transmits the sound data read from the memory 3 to the first volume control unit 31. As a result, the first volume control unit 31 performs control such as outputting sound based on the sound data read from the memory 3 from the first speaker 61 at a volume corresponding to the volume setting information and stopping the output of the sound. Furthermore, the control unit 1 determines that the output of sound from the first speaker 61 has started or ended by receiving information on the control result for the first speaker 61 from the first volume control unit 31. Moreover, the control unit 1 acquires directional angle information of the first speaker 61. In the case of this embodiment, directional angle information of the first speaker 61 is stored in the memory 3, and the control unit 1 acquires the directional angle information read from the memory 3. Furthermore, information on the maximum sound pressure in the first speaker 61 according to the distance is also stored in the memory 3, and the control unit 1 obtains the information on the maximum sound pressure read from the memory 3.

[0034] The control unit 1 also transmits control information specifying the pan amount and tilt amount of the first camera head 71 to the first camera head control unit 21. As a result, the first camera head control unit 21 performs pan drive control and tilt drive control for the first camera head 71. Furthermore, the control unit 1 receives information on the pan control and tilt control results for the first camera head 71 from the first camera head control unit 21, and determines the current pan amount and tilt amount of the first camera head 71 based on the received pan control and tilt control results. As the control unit 1 also acquires the zoom amount of the first imaging unit 41 as described above, it determines the angle of view and the shooting direction of the first imaging unit 41 based on the zoom amount and the pan amount and tilt amount of the first camera head 71. Similarly, the control unit 1 also acquires the directional angle information of the first speaker 61 as described above, and determines the range of the directional angle and the sound output direction of the first speaker 61 based on the directional angle information and the pan amount and tilt amount of the first camera head 71. This allows the control unit 1 to determine the correspondence between the angle of view and shooting direction of the first imaging unit 41 and the range of the directivity angle and the sound output direction of the first speaker 61. Furthermore, from information on the maximum sound pressure according to distance at the first speaker 61, the control unit 1 can determine the distance at which the output sound of the first speaker 61 can reach the target subject as an effective voice to call out to the subject.

[0035] Moreover, the control unit 1 transmits information such as the start and end of the imaging operation in the second imaging unit 42 and the zoom amount to the second imaging control unit 12 as control information for the second imaging control unit 12. As a result, the second imaging control unit 12 controls the start and end of the imaging operation for the second imaging unit 42, controls the zoom, etc. Furthermore, the control unit 1 receives information on the zoom control result in the second imaging unit 42 from the second imaging control unit 12, and determines the zoom amount, i.e., the angle of view, of the second imaging unit 42 at the current time based on the received zoom control result.

[0036] The control unit 1 also transmits control information specifying the tilt amount of the second tripod head 72 to the second tripod head control unit 22, and transmits control information specifying the pan amount of the fourth tripod head 74 to the fourth tripod head control unit 24. As a result, the second tripod head control unit 22 performs tilt drive control for the second tripod head 72, and the fourth tripod head control unit 24 performs pan drive control for the fourth tripod head 74. Furthermore, the control unit 1 receives information on the tilt control result for the second tripod head 72 from the second tripod head control unit 22, and determines the current tilt amount of the first tripod head 71 based on the received tilt control result. Similarly, the control unit 1 receives information on the pan control result for the fourth tripod head 74 from the fourth tripod head control unit 24, and determines the current pan amount of the fourth tripod head 74 based on the received pan control result. Then, as described above, the control unit 1 acquires the zoom amount of the second imaging unit 42, and determines the angle of view and shooting direction of the second imaging unit 42 based on the zoom amount, the tilt amount of the second tripod head 72, and the pan amount of the fourth tripod head 74.

[0037] The control unit 1 also transmits information such as the start and end of the audio output operation in the second speaker 62, volume setting information, and audio output from the second speaker 62 (audio data corresponding to the content of the call to the target subject) to the second volume control unit 32. The audio output from the second speaker 62 (audio corresponding to the content of the call) is also stored in the memory 3, and the control unit 1 transmits the audio data read from the memory 3 to the second volume control unit 32. As a result, the second volume control unit 32 performs control such as outputting audio based on the audio data read from the memory 3 from the second speaker 62 at a volume corresponding to the volume setting information and stopping the output of the audio. Furthermore, the control unit 1 determines that the output of audio from the second speaker 62 has started or ended by receiving information on the control result for the second speaker 62 from the second volume control unit 32. The control unit 1 also acquires directional angle information of the second speaker 62. In the case of this embodiment, the directional angle information of the second speaker 62 is stored in the memory 3, and the control unit 1 acquires the directional angle information read from the memory 3. Moreover, information on the maximum sound pressure in the second speaker 62 according to the distance is also stored in the memory 3, and the control unit 1 obtains the information on the maximum sound pressure from the memory 3.

[0038] The control unit 1 also receives information on the tilt control result for the third camera head 73 from the third camera head control unit 23, and determines the current tilt amount of the third camera head 73 based on the received tilt control result. As described above, the control unit 1 also receives information on the pan control result for the fourth camera head 74 from the fourth camera head control unit 24, and determines the current pan amount of the fourth camera head 74 based on the received pan control result. Thereby, the control unit 1 determines the directivity angle range and audio output direction of the second speaker 62 based on the directivity angle information of the second speaker 62 described above, the tilt amount of the third camera head 73, and the pan amount of the fourth camera head 74. The control unit 1 can also determine the distance that the output audio of the second speaker 62 can reach as an effective audio for the target subject based on the information on the maximum sound pressure according to the distance in the second speaker 62.

[0039] Furthermore, the control unit 1 can determine the correspondence between the angle of view and the imaging direction of the second imaging unit 42 described above and the directivity angle range and the sound output direction of the second speaker 62. That is, the control unit 1 can determine whether the imaging direction of the second imaging unit 42 and the sound output direction of the second speaker 62 are approximately the same direction or whether the imaging direction of the second imaging unit 42 and the sound output direction of the second speaker 62 are different directions.

[0040] Furthermore, the control unit 1 performs a process of detecting a subject of interest from the captured image, using the image after image processing by the image processing unit 2 and information for identifying the subject of interest stored in the memory 3. The control unit 1 then performs tracking control such that the first imaging unit 41 tracks the subject of interest by controlling the drive of the first camera head driving unit 51 via the first camera head control unit 21. Furthermore, at the same time, the control unit 1 controls each unit to output a sound from at least one of the first speaker 61 and the second speaker 62 and to speak to the subject of interest.

[0041] Hereinafter, the information processing in the NW camera of this embodiment will be described with an assumed example of a situation where tracking control of a target subject is performed and voice output of a call is performed from the first speaker 61 and the second speaker 62 to the target subject. In this embodiment, a person is taken as the subject, and an example is given in which a person of interest is detected as a target subject (hereinafter, referred to as a target person), and a call is performed by voice output while the target person is tracked. In addition, this embodiment will be described with an assumed example in which a person of interest shown in an image captured by the second imaging unit 42, which is a wide-angle camera, is tracked by the first imaging unit 41 and the PTZ camera of the first pan head 71, and voice is output from at least one of the first speaker 61 and the second speaker 62.

[0042] In the NW camera of this embodiment, as shown in FIG. 2, both the first imaging unit 41 and the first speaker 61 are mounted on the first pan head 71, and the shooting direction of the first imaging unit 41 and the audio output direction of the first speaker 61 are in the same direction. Therefore, when the first imaging unit 41 tracks a person of interest by driving the first pan head 71, the orientations of the first imaging unit 41 and the first speaker 61 are in the same direction. On the other hand, the shooting direction of the second imaging unit 42, which is a wide-angle camera, is usually not changed even if the first imaging unit 41 tracks a person of interest. In other words, even if the first imaging unit 41 tracks a person of interest, the second pan head 72 on which the second imaging unit 42 is mounted and the fourth pan head 74 on which the second pan head 72 is mounted are not driven. Similarly, the third pan head 73 mounted on the fourth pan head 74 is not driven, and therefore the orientation of the second speaker 62 mounted on the third pan head 73, i.e., the audio output direction, is usually not changed either. As described above, the second imaging unit 42 and the second speaker 62 can be moved independently in the tilt direction by the second pan head 72 or the third pan head 73, but the tilt directions of the second imaging unit 42 and the second speaker 62 are usually controlled to be the same.

[0043] In this way, the orientation of the first imaging unit 41 and the first speaker 61 is changed successively as the person of interest is tracked, whereas the orientation of the second imaging unit 42 and the second speaker 62 is not changed. For this reason, the orientation of the first imaging unit 41 and the first speaker 61 is often different from that of the second imaging unit 42 and the second speaker 62, and depending on the orientation, it may not be possible to effectively call out to the person of interest by voice output. In this case, not only is it impossible to effectively call out to the person of interest, but the voice of the call may affect (annoy) other people other than the person of interest. Furthermore, for example, even if the first speaker 61 and the second speaker 62 are facing the direction of the person of interest, depending on the distance to the person of interest, the voice of the call may not be heard sufficiently by the person of interest (effective call may not be made).

[0044] The control unit 1 then controls the output volume of the first speaker 61 and the second speaker 62 based on the angle of view and shooting direction of the first imaging unit 41 and the second imaging unit 42, the sound output direction, directional angle range and maximum sound pressure according to the distance of the first speaker 61 and the second speaker 62, and the distance to the person of interest.

[0045] Fig. 3 is a flowchart showing the flow of information processing performed by the control unit 1 according to the present embodiment shown in Fig. 1 and Fig. 2. Hereinafter, the positional relationship between the shooting direction, the sound output direction, and the target subject shown in Fig. 4 will be described as an example.

[0046] In step S1, the control unit 1 determines whether or not a person of interest, which is a subject of interest, is captured in both images by analyzing both images acquired by the first imaging unit 41 and the second imaging unit 42 and processed by the image processing unit 2. If the control unit 1 determines that a person of interest is captured in both images acquired by the first imaging unit 41 and the second imaging unit 42, it proceeds to step S2, and if not, it proceeds to step S4.

[0047] Here, the case where the process proceeds to step S2 is considered to be a case where the shooting direction and angle of view of the first imaging unit 41 and the second imaging unit 42 and the positional relationship of the person of interest are, for example, as shown in Fig. 4(a). In Fig. 4(a) to Fig. 4(c), the range enclosed by the dashed line 81 represents the shooting direction and angle of view of the first imaging unit 41, and the range enclosed by the dotted line 82 represents the shooting direction and angle of view of the second imaging unit 42. In other words, the case where the process proceeds to step S2 is considered to be a case where, as shown in Fig. 4(a), the person of interest 85 appears in both images acquired by the first imaging unit 41 and the second imaging unit 42, and the shooting directions of the first imaging unit 41 and the second imaging unit 42 are generally the same.

[0048] When the process proceeds to step S2, the control unit 1 acquires the distance to the person of interest (the distance from the imaging unit to the person of interest) that appears in an image acquired by, for example, the first imaging unit 41 and processed by the image processing unit 2. In the distance acquisition process of step S2, the distance is calculated by an existing technique that uses, for example, focus information of the first imaging unit 41, information by image plane phase difference AF, etc. Note that the distance may also be calculated from an image acquired by the second imaging unit 42.

[0049] Next, in step S3, the control unit 1 determines the speaker that outputs the voice according to the distance calculated in step S2. In this embodiment, the control unit 1 determines the speaker that outputs the voice according to whether the distance calculated in step S2 is within a predetermined threshold distance. As described above, the first speaker 61 is a speaker with a maximum sound pressure of 60 dB at 30 m away, and the second speaker 62 is a speaker with a maximum sound pressure of 60 dB at 50 m away, so in this embodiment, 30 m at which the maximum sound pressure of the first speaker 61 is 60 dB is used as the predetermined threshold distance. In this embodiment, if the distance to the person of interest is within 30 m, for example, the control unit 1 determines either the first speaker 61 or the second speaker 62 as the speaker that outputs the voice. Also, if the distance to the person of interest is 40 m, for example, the control unit 1 determines the second speaker 62 as the speaker that outputs the voice. That is, in the case of the positional relationship of FIG. 4(a), the speaker that outputs the sound is determined to be the first speaker 61 or the second speaker 62 if the distance to the person of interest is within the threshold distance, and the second speaker 62 if the distance is greater than the threshold distance.

[0050] As described above, the second imaging unit 42 can be moved independently in the tilt direction by the second camera head 72, and the second speaker 62 can be moved independently in the tilt direction by the third camera head 73. Therefore, when the second speaker 62 is determined as the speaker that outputs sound in step S3, the sound output direction of the second speaker 62 and the shooting direction of the second imaging unit 42 may differ in the tilt direction. Therefore, when the control unit 1 determines the second speaker 62 as the speaker that outputs sound in step S3, it acquires the tilt amount of the second camera head 72 from the second camera head control unit 22 and acquires the tilt amount of the third camera head 73 from the third camera head control unit 23. Furthermore, the control unit 1 determines the shooting direction of the second imaging unit 42 from the tilt amount of the second camera head 72 and the sound output direction of the second speaker 62 from the tilt amount of the third camera head 73, and judges whether they are approximately the same direction or not. If these directions are not the same, the control unit 1 drives the third camera-head 73 by controlling the third camera-head driving unit 53 via the third camera-head control unit 23 so that the audio output direction of the second speaker 62 becomes approximately the same direction as the imaging direction of the second imaging unit 42. In this way, when the control unit 1 determines in step S3 that the speaker that outputs audio is the second speaker 62, it performs tilt drive control as necessary to adjust the audio output direction of the second speaker 62.

[0051] For example, in step S3, when the control unit 1 determines that the speaker that outputs the sound is the first speaker 61, the control unit 1 transmits control information to the first volume control unit 31 to output the sound from the first speaker 61 at maximum volume. On the other hand, the control unit 1 transmits control information to the second volume control unit 32 to mute the second speaker 62, that is, control information to set the volume to zero. Also, for example, in step S3, when the control unit 1 determines that the speaker that outputs the sound is the second speaker 62, the control unit 1 transmits control information to the second volume control unit 32 to output the sound from the second speaker 62 at maximum volume. On the other hand, the control unit 1 transmits control information to the first volume control unit 31 to mute the first speaker 61 (control information to set the volume to zero). This enables effective calling according to the distance to the person of interest. After step S3, the control unit 1 returns the process to step S1.

[0052] When the process proceeds from step S1 to step S4 described above, the control unit 1 determines whether or not the person of interest is captured in the image acquired by the second imaging unit 42. When the control unit 1 determines that the person of interest is captured in the image acquired by the second imaging unit 42, the process proceeds to step S5, and when the control unit 1 determines that the person of interest is captured in the image acquired by the second imaging unit 42, the process proceeds to step S6.

[0053] Here, the case where the process proceeds from step S4 to step S5 is considered to be a case where the shooting direction and angle of view of the first imaging unit 41 and the second imaging unit 42 and the positional relationship of the person of interest are, for example, as shown in Fig. 4(b). In other words, the case where the process proceeds to step S5 is considered to be a case where the person of interest 85 is present within the angle of view of the second imaging unit 42 but is not present within the angle of view of the first imaging unit 41. For this reason, when the process proceeds to step S5, the control unit 1 determines that the speaker that will output the sound is the second speaker 62.

[0054] Note that even when the second speaker 62 is determined in step S5 as the speaker that outputs sound, the shooting direction of the second imaging unit 42 and the sound output direction of the second speaker 62 may differ in the tilt direction, as explained in step S3. For this reason, even when the control unit 1 determines the second speaker 62 as the speaker that outputs sound in step S5, it obtains the tilt amount of the second camera head 72 from the second camera head control unit 22 and obtains the tilt amount of the third camera head 73 from the third camera head control unit 23. Furthermore, as described above, the control unit 1 determines the shooting direction of the second imaging unit 42 from the tilt amount of the second camera head 72 and the sound output direction of the second speaker 62 from the tilt amount of the third camera head 73, and judges whether they are approximately the same direction. If these directions are not the same, the control unit 1 drives the third tripod head 73 by controlling the third tripod head drive unit 53 via the third tripod head control unit 23 so that the sound output direction of the second speaker 62 is approximately the same as the imaging direction of the second imaging unit 42.

[0055] When the control unit 1 determines in step S5 that the speaker that will output the sound is the second speaker 62, it adjusts the sound output direction by tilt drive control as necessary as described above, and then transmits control information to the second volume control unit 32 to output the sound from the second speaker 62. The control unit 1 also transmits control information to the first volume control unit 31 to mute the first speaker 61. This makes it possible to effectively call out to the person of interest using the sound output from the second speaker 62. Furthermore, since no sound is output from the first speaker 61, it is possible to prevent other people other than the person of interest from being inconvenienced. Then, after step S5, the control unit 1 returns the process to step S1.

[0056] When the process proceeds from step S4 to step S6 described above, the control unit 1 determines whether or not the person of interest is captured in the image acquired by the first imaging unit 41. When the control unit 1 determines that the person of interest is captured in the image acquired by the first imaging unit 41, the process proceeds to step S8, and when the control unit 1 determines that the person of interest is captured in the image acquired by the first imaging unit 41, the process proceeds to step S7.

[0057] Here, the case where the process proceeds from step S6 to step S7 is considered to be a case where the shooting direction and angle of view of the first imaging unit 41 and the second imaging unit 42, and the positional relationship of the person of interest are, for example, as shown in FIG. 4(c). That is, when the process proceeds to step S7, it is considered that the person of interest 85 exists within the angle of view of the first imaging unit 41, but does not exist within the angle of view of the second imaging unit 42. Therefore, when the process proceeds to step S7, the control unit 1 determines the speaker that outputs the sound to be the first speaker 61. Then, the control unit 1 transmits control information to the first volume control unit 31 to output the sound from the first speaker 61. This makes it possible to call out to the person of interest. Also, since no sound is output from the second speaker 62, it is possible to prevent other people other than the person of interest from being inconvenienced. Then, after step S7, the control unit 1 returns the process to step S1.

[0058] On the other hand, if the process proceeds to step S8, the control unit 1 transmits control information for muting the first speaker 61 and the second speaker 62 to the first volume control unit 31 and the second volume control unit 32. Here, when the process proceeds from step S6 to step S8, it is considered that the shooting direction and angle of view of the first imaging unit 41 and the second imaging unit 42 and the positional relationship of the person of interest are not, for example, any of those shown in FIG. 4(a) to FIG. 4(c). In other words, when the process proceeds to step S8, it is considered that the person of interest 85 does not exist in the shooting direction and angle of view of either the first imaging unit 41 or the second imaging unit 42. Therefore, when the process proceeds to step S8, the control unit 1 transmits control information for muting the first speaker 61 and the second speaker 62 to the first volume control unit 31 and the second volume control unit 32. This makes it possible to prevent people and the like existing around the NW camera of this embodiment from being inconvenienced. After the process of step S8 is performed, the control unit 1 ends the process of the flowchart of FIG. 3.

[0059] 3, the process returns to step S1 after the process of step S3, S5, or S7. Therefore, for example, when the person of interest moves from the shooting range of either the first imaging unit 41 or the second imaging unit 42 to the shooting range of the other, the speaker that outputs the sound is determined based on the shooting direction of the imaging unit after the movement. That is, when the imaging unit capturing the image in which the person of interest appears is switched, the control unit 1 controls so that sound is output from the speaker with the sound output direction according to the shooting direction of the imaging unit capturing the image in which the person of interest appears, and the other speakers are muted.

[0060] As described above, in the first embodiment, the volume of the first speaker 61 and the second speaker 62 is controlled based on the shooting direction and angle of view of the first imaging section 41 and the second imaging section 42, and the position of the subject of interest captured in the images acquired by the first imaging section 41 and the second imaging section 42. As a result, according to the first embodiment, it is possible to realize effective audio output for the subject of interest and reduce the influence of the audio output on subjects other than the subject of interest.

[0061] Second Embodiment In the first embodiment described above, an example was given in which audio is output from a speaker oriented in a direction corresponding to the shooting direction of the imaging unit that captured the image showing the target subject. In the second embodiment described below, an example is described in which, when a sufficient volume cannot be ensured by audio output only from the speaker corresponding to the imaging unit that captured the image showing the target subject, audio is output by directing other speakers toward the target subject. Note that the configuration of the NW camera in the second embodiment is the same as the example shown in FIG. 1, and therefore illustration and description thereof are omitted. The following description will focus on the configuration and processing that are different from the first embodiment.

[0062] Fig. 5 is a flowchart showing the flow of information processing in the control unit 1 of the second embodiment. In the flowchart of Fig. 5, the same processing steps as those in the flowchart of Fig. 3 are given the same reference numbers as those in Fig. 3, and their explanations will be omitted as appropriate.

[0063] In the second embodiment, in step S6, if the control unit 1 determines that the person of interest is captured in the image acquired by the first imaging unit 41, the process proceeds to step S10, and if not, the process proceeds to step S8. Note that the process proceeds from step S6 to step S10 when, as described above, the shooting directions and angles of view of the first imaging unit 41 and the second imaging unit 42 and the positional relationship of the person of interest are as shown in FIG. 4(c), for example.

[0064] When the process proceeds to step S10, the control unit 1 acquires the distance to the person of interest appearing in the image acquired by the first imaging unit 41 and processed by the image processing unit 2. The distance is acquired in the same manner as in the process of step S2. After the distance acquisition process in step S10, the control unit 1 advances the process to step S11.

[0065] When the process proceeds to step S11, the control unit 1 judges whether or not the volume of the voice to be called to the person of interest at the distance calculated in step S10 is sufficient by only the first speaker 61 mounted on the first camera platform 71 together with the first imaging unit 41. For example, the control unit 1 judges whether or not the sound pressure of the output sound from the first speaker 61 at the distance to the person of interest is equal to or greater than a predetermined threshold sound pressure. The predetermined threshold sound pressure is a sound pressure at which the person of interest can hear the voice, and when a voice is issued to warn the person of interest, for example, the predetermined threshold sound pressure is determined in advance as a sound pressure at which the person of interest can hear the voice that is being warned. For example, when the predetermined threshold sound pressure is 60 dB, the maximum sound pressure of the first speaker 61 at a distance of 30 m is 60 dB, so the control unit 1 judges that the volume of the voice to be called to the person of interest is sufficient by only the first speaker 61 if the distance to the person of interest is within 30 m. On the other hand, if the distance to the person of interest is, for example, 40 m, the control unit 1 determines that the volume of the first speaker 61 alone is insufficient for calling out to the person of interest. If the control unit 1 determines that the volume of the first speaker 61 is sufficient for calling out to the person of interest at the distance calculated in step S10, the process proceeds to step S12, and if not, the process proceeds to step S13.

[0066] When the process proceeds from step S11 to step S12, the control unit 1 determines the speaker that outputs the sound to be the first speaker 61, as in step S7 of FIG. 3 described above. Then, the control unit 1 transmits control information to the first volume control unit 31 to output the sound from the first speaker 61. This makes it possible to call out to the person of interest. Also, since no sound is output from the second speaker 62, it is possible to prevent other people other than the person of interest from being inconvenienced. Then, after step S12, the control unit 1 returns the process to step S1.

[0067] On the other hand, when the process proceeds to step S13, the control unit 1 transmits control information to the third tripod head control unit 23 and the fourth tripod head control unit 24 to move the orientation of the second speaker 62 toward the audio output direction of the first speaker 61. As described above, since the first speaker 61 is mounted on the first tripod head 71 together with the first imaging unit 41, the control unit 1 obtains the pan and tilt amounts of the first tripod head 71 from the first tripod head control unit 21 and determines the current shooting direction of the first imaging unit 41 based on the amounts. Then, the control unit 1 controls the third tripod head control unit 23 and the fourth tripod head control unit 24 so that the orientation of the second speaker 62 is approximately the same as the shooting direction of the first imaging unit 41. Then, the third tripod head control unit 23 and the fourth tripod head control unit 24 drive and control the third tripod head drive unit 53 and the fourth tripod head drive unit 54, so that the orientation of the second speaker 62 moves toward the audio output direction of the first speaker 61.

[0068] Next, in step S14, the control unit 1 sets the volume of sounds to be output from the first speaker 61 and the second speaker 62, based on the distance to the person of interest calculated in step S11. Furthermore, the control unit 1 transmits control information to the first volume control unit 31 so as to output sound from the first speaker 61 at the volume set for the first speaker 61. Similarly, the control unit 1 transmits control information to the second volume control unit 32 so as to output sound from the second speaker 62 at the volume set for the second speaker 62.

[0069] Here, if the predetermined threshold sound pressure at which the person of interest can hear the voice of the call is 60 dB and the distance to the person of interest is 40 m, the control unit 1 sets the output volume of the first speaker 61 to the maximum volume, that is, the maximum volume at which the maximum sound pressure at 30 m away is 60 dB. On the other hand, the control unit 1 sets the volume of the second speaker 62 to the maximum volume that can be combined with the output sound of the first speaker 61 to be 60 dB or more at 40 m away. That is, as described above, the second speaker 62 can output sound at a volume at which the maximum sound pressure at 50 m away is 60 dB, but in this example, the control unit 1 sets the volume of the second speaker 62 to be 60 dB at 30 m away. When the volumes of the first speaker 61 and the second speaker 62 are set in this way and sound is output from the first speaker 61 and the second speaker 62, the sound pressure at 40 m away where the person of interest is located is about 60.5 dB. After step S14, the control unit 1 returns the process to step S1.

[0070] 5, the process returns to step S1 after the processing of steps S3, S5, S12, and S14. Therefore, for example, when a person of interest moves from the shooting range of either the first imaging unit 41 or the second imaging unit 42 to the shooting range of the other, the speaker that outputs sound is determined based on the shooting direction of the imaging unit after the movement. That is, in the second embodiment, similarly to the first embodiment, the control unit 1 controls the volume of each speaker in response to switching of the imaging unit that is acquiring an image in which the person of interest is shown.

[0071] Furthermore, in step S3, if the distance to the person of interest calculated in step S2 is a distance at which a predetermined threshold sound pressure cannot be obtained even if sound is output from either the first speaker 61 or the second speaker 62, the control unit 1 may perform control similar to that in step S14. That is, in step S3, similar to step S14, the control unit 1 may perform control so as to output sound from both the first speaker 61 and the second speaker 62 based on the distance to the person of interest obtained in step S2. Note that such processing is also applicable to step S3 in FIG. 3 of the first embodiment described above.

[0072] As described above, in the second embodiment, if the volume of sound output from one speaker is insufficient to reach the subject of interest, the other speakers are also directed toward the subject of interest and sound is output from those speakers, thereby achieving effective sound output to the subject of interest in the second embodiment.

[0073] In the above-described embodiment, an example was given in which there are two imaging units and two speakers, but this is not limited thereto, and there may be three or more imaging units and speakers. The number of speakers corresponding to the imaging units is also not limited to one, and multiple speakers may be provided for one imaging unit. In the above-described embodiment, a configuration was given in which the first imaging unit and the first speaker are mounted on the first camera platform, and the orientations of the first imaging unit and the first speaker move in the same direction, but this is not limited thereto, and the orientations of the first imaging unit and the first speaker may move independently.

[0074] FIG. 6 is a diagram showing a hardware configuration relating to the control unit 1 of the NW camera according to the embodiment described above and its peripheral configuration. The CPU 91 realizes the functions of the image processing unit 2, the first imaging control unit 11, the first camera-head control unit 21, the second imaging control unit 12, the second camera-head control unit 22, the third camera-head control unit 23, the fourth camera-head control unit 24, the first volume control unit 31, and the second volume control unit 32 shown in Fig. 1. The CPU 91 realizes the processes related to each of these functional units by expanding the information processing program according to this embodiment stored in the ROM 92 into the RAM 93 and executing it. The ROM 92 and the RAM 93 realize the functions of the memory 3 shown in Fig. 1.

[0075] The communication circuit 94 is a communication device that realizes the communication unit 4 shown in Fig. 1. The camera I / F 95 is an interface that exchanges image data and control information between the first imaging unit 41 and the second imaging unit 42 shown in Fig. 1 and Fig. 2. The speaker I / F 96 is an interface that exchanges audio data and control information between the first speaker 61 and the second speaker 62 shown in Fig. 1 and Fig. 2. The tripod head I / F 97 is an interface that exchanges pan and tilt control information and information indicating the results of the tripod head control for the first tripod head drive unit 51, the second tripod head drive unit 52, the third tripod head drive unit 53, and the fourth tripod head drive unit 54 shown in Fig. 1 and Fig. 2.

[0076] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main characteristics.

[0077] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) a determination means for determining whether or not a target object is present in a first image acquired by a first imaging device and a second image acquired by a second imaging device; a control means for controlling a volume of a first speaker corresponding to the first imaging device and a second speaker corresponding to the second imaging device; having The control means When the determination means determines that the target object is present in both the first image and the second image, a first control is performed; When the determination means determines that the target object is present in either the first image or the second image, a second control different from the first control is performed; An information processing device characterized in that, when the determination means determines that the subject of interest is not present in either the first image or the second image, a third control different from the first control and the second control is performed. (Configuration 2) The information processing device described in configuration 1, characterized in that when the determination means determines that the subject of interest is present in either the first image or the second image, the control means performs the second control of outputting sound from a speaker corresponding to an imaging device that is capturing the subject of interest and muting the volume of a speaker corresponding to an imaging device that is not capturing the subject of interest. (Configuration 3) The information processing device described in configuration 1 or 2, characterized in that the control means performs the third control of muting the volume of the first and second speakers when the determination means determines that the subject of interest is not present in either the first image or the second image. (Configuration 4) a distance acquisition means for acquiring a distance to the target object, The information processing device described in any one of configurations 1 to 3, characterized in that when the determination means determines that the subject of interest is present in both the first image and the second image, the control means performs the first control to determine the speaker that will output audio to the first speaker or the second speaker based on the distance to the subject of interest. (Configuration 5) The first speaker and the second speaker are speakers at different distances at which the same sound pressure can be obtained, The information processing device described in configuration 4, characterized in that when both the first imaging device and the second imaging device are imaging the subject of interest, the control means determines, among the first speaker or the second speaker, the speaker with the loudest maximum volume as the speaker to output the audio. (Configuration 6) The first speaker and the second speaker are speakers at different distances at which the same sound pressure can be obtained, The information processing device described in configuration 4, characterized in that when both the first imaging device and the second imaging device are imaging the subject of interest, the control means determines, from the first speaker or the second speaker, a speaker that can output sound of a predetermined threshold sound pressure or more at a distance to the subject of interest, as the speaker to output the sound. (Configuration 7) The information processing device described in configuration 4, characterized in that when both the first imaging device and the second imaging device are imaging the subject of interest, if there is no speaker among the first speaker or the second speaker that can output audio of a predetermined threshold sound pressure or higher within a distance to the subject of interest, the control means determines the first speaker or the second speaker as the speaker that will output the audio. (Configuration 8) The information processing device according to configuration 7, wherein the control means controls the volume of each of the first speaker and the second speaker so that the combined volume of the output sound of the first speaker or the output sound of the second speaker is equal to or greater than the predetermined threshold sound pressure at the distance to the subject of interest. (Configuration 9) The information processing device described in configuration 6, characterized in that when the first speaker corresponding to the first imaging device imaging the subject of interest has a lower sound pressure than a predetermined threshold sound pressure at a distance to the subject of interest, the control means controls the sound output direction of the second speaker to align the sound output direction of the second speaker with the shooting direction of the first imaging device imaging the subject of interest. (Configuration 10) The information processing device described in configuration 9, wherein the control means controls the volume of each of the output sound from the first speaker, which is lower than the predetermined threshold sound pressure, and the output sound from the second speaker, whose sound output direction is controlled, so that the combined volume of the output sound from the first speaker, which is lower than the predetermined threshold sound pressure, is equal to or greater than the predetermined threshold sound pressure at the distance to the subject of interest. (Configuration 11) The information processing device described in any one of configurations 1 to 10, characterized in that when the subject of interest moves from the shooting range of the first imaging device to the shooting range of the second imaging device, the control means controls the volume of the first speaker and the second speaker based on the shooting direction of the second imaging device. (Configuration 12) Both the first imaging device and the first speaker are mounted on a first camera platform that is movable in pan and tilt directions; the second imaging device is mounted on a second camera platform that is movable in a tilt direction; the second speaker is mounted on a third platform that can be driven in a tilt direction; The second head and the third head are both mounted on a fourth head that can be driven in a panning direction; The control means determining an imaging direction of the first imaging device and an audio output direction of the first speaker based on the pan and tilt amounts of the first camera head; determining a photographing direction of the second imaging device based on a tilt amount of the second camera head and a pan amount of the fourth camera head; 12. The information processing device according to any one of configurations 1 to 11, wherein the audio output direction of the second speaker is determined based on a tilt amount of the third pan head and a pan amount of the fourth pan head. (Configuration 13) the second imaging device is an imaging device having a wider angle of view than the first imaging device, 13. The information processing device according to configuration 12, further comprising a tracking control means for controlling the first imaging device so that the first imaging device tracks a subject of interest in a captured image captured by the second imaging device. (Method 1) a determination step of determining whether or not a subject of interest is present in a first image acquired by a first imaging device and a second image acquired by a second imaging device; a control step of controlling a volume of a first speaker corresponding to the first imaging device and a second speaker corresponding to the second imaging device; having In the control step, when it is determined in the determination step that the target subject is present in both the first image and the second image, a first control is performed, When it is determined in the determination step that the target subject is present in either the first image or the second image, a second control different from the first control is performed; An information processing method characterized in that, when it is determined in the judgment process that the subject of interest is not present in either the first image or the second image, a third control different from the first control and the second control is performed. (Program 1) A program for causing a computer to function as the information processing device according to any one of configurations 1 to 13. [Explanation of symbols]

[0078] 1: first imaging unit, 2: image processing unit, 3: memory, 11: first imaging control unit, 12: second imaging control unit, 21: first pan head control unit, 22: second pan head control unit, 23: third pan head control unit, 24: fourth pan head control unit, 31: first volume control unit, 32: second volume control unit

Claims

1. a determination means for determining whether or not a target object is present in a first image acquired by a first imaging device and a second image acquired by a second imaging device; a control means for controlling a volume of a first speaker corresponding to the first imaging device and a second speaker corresponding to the second imaging device; having The control means When the determination means determines that the target object is present in both the first image and the second image, a first control is performed; When the determination means determines that the target object is present in either the first image or the second image, a second control different from the first control is performed; An information processing device characterized in that, when the determination means determines that the subject of interest is not present in either the first image or the second image, a third control different from the first control and the second control is performed.

2. The information processing device according to claim 1, characterized in that, when the determination means determines that the subject of interest is present in either the first image or the second image, the control means performs the second control of outputting sound from a speaker corresponding to an imaging device that is capturing the subject of interest and muting the volume of a speaker corresponding to an imaging device that is not capturing the subject of interest.

3. The information processing device according to claim 1, characterized in that the control means performs the third control of muting the volume of the first and second speakers when the determination means determines that the subject of interest is not present in either the first image or the second image.

4. a distance acquisition means for acquiring a distance to the target object, The information processing device according to claim 1, characterized in that, when the determination means determines that the subject of interest is present in both the first image and the second image, the control means performs the first control to determine the speaker that will output audio to the first speaker or the second speaker based on the distance to the subject of interest.

5. The first speaker and the second speaker are speakers at different distances at which the same sound pressure can be obtained, The information processing device according to claim 4, characterized in that, when both the first imaging device and the second imaging device are imaging the subject of interest, the control means determines, among the first speaker or the second speaker, the speaker with the loudest maximum volume as the speaker to output the audio.

6. The first speaker and the second speaker are speakers at different distances at which the same sound pressure can be obtained, The information processing device according to claim 4, characterized in that, when both the first imaging device and the second imaging device are imaging the subject of interest, the control means determines, among the first speaker or the second speaker, the speaker capable of outputting sound of a predetermined threshold sound pressure or more at a distance to the subject of interest as the speaker to output the sound.

7. The information processing device according to claim 4, characterized in that when both the first imaging device and the second imaging device are imaging the subject of interest, if there is no speaker among the first speaker or the second speaker that is capable of outputting audio of a predetermined threshold sound pressure or higher within a distance to the subject of interest, the control means determines the first speaker or the second speaker as the speaker that will output the audio.

8. The information processing device according to claim 7, characterized in that the control means controls the volume of each speaker so that the combined volume of the output sound from the first speaker or the output sound from the second speaker is equal to or greater than the predetermined threshold sound pressure at a distance to the subject of interest.

9. The information processing device according to claim 6, characterized in that when the first speaker corresponding to the first imaging device imaging the subject of interest has a sound pressure lower than a predetermined threshold sound pressure at a distance to the subject of interest, the control means controls the sound output direction of the second speaker so as to align the sound output direction of the second speaker with the shooting direction of the first imaging device imaging the subject of interest.

10. The information processing device described in claim 9, characterized in that the control means controls the volume of each of the output sound from the first speaker, which is lower than the predetermined threshold sound pressure, and the output sound from the second speaker, whose sound output direction is controlled, so that the combined volume of the output sound from the first speaker, which is lower than the predetermined threshold sound pressure, is equal to or greater than the predetermined threshold sound pressure at the distance to the subject of interest.

11. The information processing device according to claim 1, characterized in that the control means controls the volume of the first speaker and the second speaker based on the shooting direction of the second imaging device when the subject of interest moves from the shooting range of the first imaging device to the shooting range of the second imaging device.

12. Both the first imaging device and the first speaker are mounted on a first camera platform that is movable in pan and tilt directions; the second imaging device is mounted on a second camera platform that is movable in a tilt direction; the second speaker is mounted on a third platform that can be driven in a tilt direction; The second head and the third head are both mounted on a fourth head that can be driven in a panning direction; The control means determining an imaging direction of the first imaging device and an audio output direction of the first speaker based on the pan and tilt amounts of the first camera head; determining a photographing direction of the second imaging device based on a tilt amount of the second camera head and a pan amount of the fourth camera head; 2. The information processing apparatus according to claim 1, wherein the audio output direction of the second speaker is determined based on a tilt amount of the third pan head and a pan amount of the fourth pan head.

13. the second imaging device is an imaging device having a wider angle of view than the first imaging device, 13. The information processing apparatus according to claim 12, further comprising a tracking control means for controlling the first imaging device so that the first imaging device tracks a subject of interest in a photographed image captured by the second imaging device.

14. a determination step of determining whether or not a subject of interest is present in a first image acquired by a first imaging device and a second image acquired by a second imaging device; a control step of controlling a volume of a first speaker corresponding to the first imaging device and a second speaker corresponding to the second imaging device; having In the control step, when it is determined in the determination step that the target subject is present in both the first image and the second image, a first control is performed, When it is determined in the determination step that the target subject is present in either the first image or the second image, a second control different from the first control is performed; An information processing method characterized in that, when it is determined in the judgment process that the subject of interest is not present in either the first image or the second image, a third control different from the first control and the second control is performed.

15. Computer, a determination means for determining whether or not a target object is present in a first image acquired by a first imaging device and a second image acquired by a second imaging device; a control means for controlling a volume of a first speaker corresponding to the first imaging device and a second speaker corresponding to the second imaging device; having the control means performs a first control when the determination means determines that the target subject is present in both the first image and the second image, When the determination means determines that the target object is present in either the first image or the second image, a second control different from the first control is performed; A program that causes the information processing device to function as an information processing device performing a third control different from the first control and the second control when the determination means determines that the subject of interest is not present in either the first image or the second image.

Images