Information processing apparatus, information processing method, and program

Abstract

To enable realization of effective voice output and reduction of the influence of the voice output.SOLUTION: An information processing apparatus has first control means that controls a photographing direction of a first imaging apparatus, second control means that controls the direction of a speaker, and sound volume control means that controls the sound volume of the speaker. The information processing apparatus sets the sound volume of the speaker controlled by the sound volume control means on the basis of the photographing direction of the first imaging apparatus and the direction of the speaker.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing technique used to control an apparatus that captures images and outputs audio. [Background technology]

[0002] Some network cameras are equipped with so-called pan, tilt, and zoom functions. Hereinafter, a network camera will be referred to as a NW camera, pan, tilt, and zoom will be referred to as PTZ, and a camera with PTZ functions will be referred to as a PTZ camera. Some NW cameras are equipped with a wide-angle camera capable of capturing images with a wide angle of view like a panoramic camera, and a speaker capable of outputting audio, in addition to the PTZ camera. With this NW camera, for example, a subject of interest (e.g., a person) captured in an image captured by the wide-angle camera can be tracked by the PTZ camera, and audio can be output from the speaker, allowing the subject of interest (person) to be spoken to (warned, called attention, etc.).

[0003] Patent document 1 discloses a technology that, in a system having a microphone array and multiple speakers, estimates the position of a speaker using the microphone array and selects the speaker that will emit sound based on the estimated position, thereby enabling audio output in the direction of a specific person. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-158134 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the case of the network camera that is capable of audio output, the direction of the speaker's audio output may shift from the direction of the target subject being tracked by the PTZ camera. In this case, not only will it be impossible to effectively call out to the target subject, but the voice of the call may affect other subjects other than the target subject.

[0006] Therefore, an object of the present invention is to enable effective audio output and reduce the influence of the audio output. [Means for solving the problem]

[0007] The information processing device of the present invention is characterized by having a first control means for controlling the shooting direction of a first imaging device, a second control means for controlling the orientation of a speaker, a volume control means for controlling the volume of the speaker, and a setting means for setting the volume of the speaker controlled by the volume control means based on the shooting direction of the first imaging device and the orientation of the speaker. Effect of the Invention

[0008] According to the present invention, it is possible to realize effective audio output and reduce the influence of the audio output. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional configuration of a network camera including an information processing device. [Diagram 2] FIG. 1 is a diagram showing an example of a schematic external configuration of a network camera. [Diagram 3] 11A and 11B are diagrams illustrating an example of control of a direction of audio output from a speaker. [Figure 4] 4 is a flowchart showing a flow of information processing in the first embodiment. [Diagram 5] 10 is a flowchart showing a flow of information processing in a second embodiment. [Figure 6] 13 is a flowchart showing a flow of information processing in a third embodiment. [Figure 7]4 is a diagram showing the relationship between the angle of view of a first imaging unit and the directivity angle range of a speaker. FIG. [Figure 8] 2 is a diagram showing a hardware configuration of a control unit and its peripheral components. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (conditions of use, environment of use, etc.). In each of the following embodiments, the same or similar configurations and processing steps are given the same reference symbols, and duplicated explanations will be omitted.

[0011] First Embodiment Fig. 1 is a block diagram showing an example of the functional configuration of a main part of a network camera (hereinafter referred to as a NW camera) as an example of an information processing device according to an embodiment, and Fig. 2 is a diagram showing an example of a schematic external configuration of the NW camera according to the embodiment. In the NW camera of this embodiment, the first imaging unit 1 is an imaging device having an imaging optical system with a zoom function and an imaging element. The imaging optical system is composed of a plurality of lenses, an aperture, and an optical filter such as an infrared cut filter. The imaging element is composed of a photoelectric conversion element such as a CCD or a CMOS. CCD is an abbreviation for Charge Coupled Device, and CMOS is an abbreviation for Complementary Metal Oxide Semiconductor.

[0012] The first imaging control unit 4 controls the imaging operation of the first imaging unit 1, the zoom operation of the imaging optical system, the output operation of the captured image, and the like. The first imaging control unit 4 also determines the angle of view of the first imaging unit 1 from the zoom amount of the imaging optical system of the first imaging unit 1. The first imaging unit 1 is mounted on a first camera platform 21 shown in FIG. 2, and the first camera platform 21 is a pan-tilt camera platform that can be driven in the pan and tilt directions, and is driven in the pan and tilt directions by a first camera platform driving unit 14. The first camera platform driving unit 14 is driven in the pan and tilt directions by, for example, a motor. In this way, the imaging optical system of the first imaging unit 1 is capable of zooming, and the first camera platform 21 is a pan-tilt camera platform, so that the first imaging unit 1 and the first camera platform 21 constitute a PTZ camera that can pan, tilt, and zoom.

[0013] The first camera-head control unit 5 drives the first camera-head 21 in the pan direction and tilt direction by controlling the drive of the first camera-head driving unit 14 under the control of the control unit 8 described later. That is, in this embodiment, the first camera-head control unit 5 functions as an imaging direction control unit that changes and controls the orientation of the first imaging unit 1 (i.e., the imaging direction) in the pan and tilt directions. In addition, the first camera-head control unit 5 transmits the amount of drive of the first camera-head 21 by the first camera-head driving unit 14 or the output of an encoder or the like provided on the first camera-head 21 to the control unit 8 described later as information indicating the results of the pan control and tilt control of the first camera-head 21.

[0014] The second imaging unit 7 is an imaging device having an imaging optical system and an imaging element capable of capturing an image with a wider angle of view than the first imaging unit 1. The imaging optical system of the second imaging unit 7 is composed of a plurality of lenses, an aperture, and an optical filter such as an infrared cut filter, and the imaging element is composed of a photoelectric conversion element such as a CCD or a CMOS. The imaging optical system of the second imaging unit 7 may have a zoom function, and the angle of view achieved by the zoom function is wider than the angle of view achieved by the imaging optical system of the first imaging unit 1. The second imaging unit 7 may also be a panoramic camera.

[0015] The second imaging control unit 9 controls the imaging operation of the second imaging unit 7, the zoom operation of the imaging optical system, the output operation of the captured image, etc. In addition, the second imaging control unit 9 determines the angle of view at the time of shooting of the second imaging unit 7 from the zoom amount in the imaging optical system of the second imaging unit 7. 2, the second camera head 22 is a tilt camera head that can be driven in a tilt direction, and is driven in the tilt direction by a second camera head drive unit 15. The second camera head drive unit 15 is driven in the tilt direction by, for example, a motor.

[0016] The second camera-head control unit 10 drives the second camera-head 22 in the tilt direction by controlling the drive of the second camera-head drive unit 15 under the control of the control unit 8 described later. That is, in this embodiment, the second camera-head control unit 10 functions as an imaging direction control unit that changes and controls the orientation (imaging direction) of the second imaging unit 7 in the tilt direction. In addition, the second camera-head control unit 10 transmits the amount of drive of the second camera-head 22 by the second camera-head drive unit 15 or the output of an encoder or the like provided on the second camera-head 22 to the control unit 8 described later as information indicating the result of the tilt control of the second camera-head 22.

[0017] The speaker 6 is an audio output device (a device in which a speaker unit is built into a housing) that has a relatively narrow directivity, and converts the audio signal input from the volume control unit 11 into vibrations in the air. The volume control unit 11 adjusts the volume of the sound output from the speaker 6 under the control of the control unit 8. The volume control unit 11 is also configured to include a D / A converter that converts the sound data to be output to the speaker 6 into an analog signal, an amplifier for driving the speaker 6, and a digital volume for adjusting the sound volume, an adjustment circuit for the gain of the amplifier, and the like. In this embodiment, the volume control unit 11 outputs sound corresponding to the sound data from the speaker 6 at a volume corresponding to the volume setting information, based on the sound data and volume setting information supplied from the control unit 8, which will be described later.

[0018] 2, the third pan head 23 is a tilt pan head that can be driven in a tilt direction, and is driven in the tilt direction by the third pan head drive unit 16. The third pan head drive unit 16 is driven in the tilt direction by, for example, a motor. The third camera-head control unit 12 drives the third camera-head 23 in the tilt direction by controlling the drive of the third camera-head driving unit 16 under the control of the control unit 8 described later. That is, in this embodiment, the third camera-head control unit 12 functions as an audio direction control unit that controls and changes the orientation of the speaker 6 (audio output direction) in the tilt direction. In addition, the third camera-head control unit 12 transmits the amount of drive of the third camera-head 23 by the third camera-head driving unit 16 or the output of an encoder or the like provided on the third camera-head 23 to the control unit 8 described later as information indicating the result of the tilt control of the third camera-head 23.

[0019] The second and third pan heads 22 and 23 are mounted on a fourth pan head 24 shown in Fig. 2, and the fourth pan head 24 is a pan head that can be driven in the pan direction, and is driven in the pan direction by a fourth pan head drive unit 17. The fourth pan head drive unit 17 is driven in the pan direction by, for example, a motor. The fourth pan head drive unit 17 is driven and controlled by a fourth pan head control unit 13, which drives and controls the fourth pan head drive unit 17 under the control of a control unit 8 described later. The fourth pan head control unit 13 also transmits the amount of drive of the fourth pan head 24 by the fourth pan head drive unit 17 or the output of an encoder or the like provided on the fourth pan head 24 to the control unit 8 described later as information indicating the result of pan control of the fourth pan head 24.

[0020] In this way, since the second head 22 is mounted on the fourth head 24, the second imaging unit 7 mounted on the second head 22 moves in the tilt direction by the second head 22 and moves in the pan direction by the fourth head. That is, in this embodiment, the second head control unit 10 and the fourth head control unit 13 controlled by the control unit 8 function as an imaging direction control unit that changes and controls the orientation of the second imaging unit 7 to the tilt and pan directions. Similarly, since the third head 23 is mounted on the fourth head 24, the speaker 6 mounted on the third head 23 moves in the tilt direction by the third head 23 and moves in the pan direction by the fourth head. That is, in this embodiment, the third head control unit 12 and the fourth head control unit 13 controlled by the control unit 8 function as an audio direction control unit that changes and controls the orientation of the speaker 6 to the tilt and pan directions. Also, the second imaging unit 7 and the speaker 6 move in the tilt direction independently by the second head 22 or the third head 23, whereas the pan direction moves in the same direction by the fourth head 24. The positional relationship between the second imaging unit 7 and the second camera platform 22 and the speaker 6 and the third camera platform 23 may be reversed from that in the example of FIG.

[0021] The image processing unit 2 performs predetermined image processing, such as color balance processing and gamma correction processing, on the image data captured by the first imaging unit 1 and the second imaging unit 7. The image processing unit 2 also generates an image file of a still image or a video in a predetermined format, such as JPEG (Joint Photographic Experts Group), H.264, or H.265, from the image data after the predetermined image processing. The image processing unit 2 then outputs the data of the image file to the memory 3 and the control unit 8.

[0022] The memory 3 has a volatile memory such as SRAM or DRAM, and a non-volatile memory such as a flash memory, and stores data output from the image processing unit 2. SRAM stands for Static Random Access Memory, and DRAM stands for Dynamic Random Access Memory. The memory 3 also stores information for identifying a subject of interest, such as face information, a person's whole-body image, body type information, clothing information, and the like. The memory 3 also stores information on the directional angle of the speaker 6, data on the sound output from the speaker 6, and the like.

[0023] The communication unit 18 is a device for performing wired or wireless communication between the NW camera of this embodiment and an external device (information terminal, etc.). The communication unit 18 transmits to the outside image data (image file) captured by the NW camera of this embodiment and processed by the image processing unit 2. The communication unit 18 also receives control data sent from an external information terminal, information on a target subject to be described later, audio data to be output from the speaker 6, etc.

[0024] The control unit 8 is a control device that controls the entire NW camera of this embodiment. The control unit 8 transmits information related to control to each of the components, namely, the image processing unit 2, the memory 3, the first imaging control unit 4, the first camera head control unit 5, the second imaging control unit 9, the second camera head control unit 10, the volume control unit 11, the third camera head control unit 12, the fourth camera head control unit 13, and the communication unit 18. The control unit 8 also receives various information and data from each of these components. The main control functions of the control unit 8 will be described below.

[0025] The control unit 8 transmits information such as the start and end of the imaging operation in the first imaging unit 1 and the zoom amount to the first imaging control unit 4 as control information for the first imaging control unit 4. As a result, the first imaging control unit 4 controls the start and end of the imaging operation for the first imaging unit 1, zoom control, and the like. In addition, the control unit 8 transmits information specifying the pan amount and tilt amount of the first camera head 21 to the first camera head control unit 5. As a result, the first camera head control unit 5 performs pan drive control and tilt drive control for the first camera head 21.

[0026] The control unit 8 also receives information on the zoom control result of the first imaging unit 1 from the first imaging control unit 4, and recognizes the current zoom amount of the first imaging unit 1 based on the received zoom control result. The control unit 8 also receives information on the pan control and tilt control result for the first camera head 21 from the first camera head control unit 5, and recognizes the current pan amount and tilt amount of the first camera head 21 based on the received pan control and tilt control result. The control unit 8 then determines the angle of view and shooting direction of the first imaging unit 1 based on the zoom amount of the first imaging unit 1 and the pan amount and tilt amount of the first camera head 21.

[0027] In addition, the control unit 8 transmits information such as the start and end of the imaging operation in the second imaging unit 7 and the zoom amount to the second imaging control unit 9 as control information for the second imaging control unit 9. As a result, the second imaging control unit 9 controls the start and end of the imaging operation for the second imaging unit 7, zoom control, and the like. In addition, the control unit 8 transmits information specifying the tilt amount of the second camera head 22 to the second camera head control unit 10, and transmits information specifying the pan amount of the fourth camera head 24 to the fourth camera head control unit 13. As a result, the second camera head control unit 10 performs tilt drive control for the second camera head 22, and the fourth camera head control unit 13 performs pan drive control for the fourth camera head 24.

[0028] The control unit 8 also receives information on the zoom control result of the second imaging unit 7 from the second imaging control unit 9, and recognizes the current zoom amount of the second imaging unit 7 based on the received zoom control result. The control unit 8 also receives information on the tilt control result for the second camera head 22 from the second camera head control unit 10, and recognizes the current tilt amount of the first camera head 21 based on the received tilt control result. Similarly, the control unit 8 receives information on the pan control result for the fourth camera head 24 from the fourth camera head control unit 13, and recognizes the current pan amount of the fourth camera head 24 based on the received pan control result. As a result, the control unit 8 determines the angle of view and shooting direction of the second imaging unit 7 based on the zoom amount of the second imaging unit 7, the tilt amount of the second camera head 22, and the pan amount of the fourth camera head 24.

[0029] Furthermore, the control unit 8 transmits information such as the start and end of the audio output operation in the speaker 6, volume setting information, and audio to be output from the speaker 6 (audio data corresponding to the content of the call to the target subject) to the volume control unit 11. In the case of this embodiment, the audio to be output from the speaker 6 (audio corresponding to the content of the call) is stored in the memory 3, and the control unit 8 transmits audio data read from the memory 3 to the volume control unit 11. As a result, the volume control unit 11 performs control such as outputting audio based on the audio data read from the memory 3 from the speaker 6 at a volume corresponding to the volume setting information, and stopping the output of the audio.

[0030] Furthermore, the control unit 8 receives information on the control result for the speaker 6 from the volume control unit 11, thereby determining that the speaker 6 has started or ended output of audio. The control unit 8 also acquires directivity angle information of the speaker 6. In the case of this embodiment, the directivity angle information of the speaker 6 is stored in the memory 3, and the control unit 8 acquires the directivity angle information read from the memory 3.

[0031] The control unit 8 also transmits information specifying the tilt amount of the third camera-to-camera head 23 to the third camera-to-camera head control unit 12. As a result, the third camera-to-camera head control unit 12 performs tilt drive control for the third camera-to-camera head 23. Furthermore, the control unit 8 receives information on the tilt control result for the third camera-to-camera head 23 from the third camera-to-camera head control unit 12, and recognizes the current tilt amount of the third camera-to-camera head 23 based on the received tilt control result. As described above, the control unit 8 also receives information on the pan control result for the fourth camera-to-camera head 24 from the fourth camera-to-camera head control unit 13, and recognizes the current pan amount of the fourth camera-to-camera head 24 based on the received pan control result. As a result, the control unit 8 determines the range and direction (audio output direction) of the directivity angle of the speaker 6 based on the directivity angle information of the speaker 6, the tilt amount of the third camera-to-camera head 23, and the pan amount of the fourth camera-to-camera head 24.

[0032] In addition, the control unit 8 sets the volume of the sound to be output from the speaker 6 based on the shooting direction and angle of view of the first imaging unit 1 determined as described above, and the directivity angle range and sound output direction of the speaker 6.

[0033] Furthermore, the control unit 8 performs a process of detecting a subject of interest from the captured image, using the image after image processing by the image processing unit 2 and information for identifying the subject of interest stored in the memory 3. The control unit 8 then performs driving control of the first camera head driving unit 14 via the first camera head control unit 5, thereby performing tracking control such that the first imaging unit 1 tracks the subject of interest, and controls each unit such that sound is output from the speaker 6 to speak to the subject of interest.

[0034] An example of a situation in which tracking control of a target subject and audio output from the speaker 6 are performed in the NW camera of this embodiment will be described below. In this embodiment, a person is taken as a subject, a person of interest is detected as a subject of interest (hereinafter, referred to as a person of interest), and an example is given in which the person of interest is called out to by voice output. Also, in this embodiment, an example is given in which a person of interest shown in an image captured by the second imaging unit 7, which is a wide-angle camera, is tracked and photographed by a PTZ camera consisting of the first imaging unit 1 and the first pan head 21, and voice is output from the speaker 6 to call out to the person of interest.

[0035] In this assumed example, depending on the positional relationship between the shooting direction and angle of view of the first imaging unit 1 tracking the person of interest and the directivity angle range and sound output direction of the speaker 6, the person of interest may not be able to hear the sound output from the speaker 6. In this case, not only will it be impossible to effectively call out to the person of interest, but there is also a risk that the voice of the call output from the speaker 6 will affect (be a nuisance to) other subjects other than the person of interest.

[0036] FIG. 3(a) is a diagram showing an example of a state in which the directional angle range and audio output direction of the speaker 6 are out of alignment with the shooting direction and angle of view of the first imaging unit 1 tracking a person of interest, that is, the angle of view and the directional angle range do not overlap. In FIG. 3(a), the range enclosed by dotted lines 31 represents the directional angle range of the speaker 6 (hereinafter referred to as directional angle range 31), and the range enclosed by dotted lines 32 represents the angle of view 32 of the first imaging unit 1. The example of FIG. 3(a) shows a state in which the directional angle range 31 of the speaker 6 is in a different direction from the angle of view 32 of the first imaging unit 1 when a person of interest is tracked by a PTZ camera consisting of the first imaging unit 1 and the first pan head 21. The example of FIG. 3(a) shows a state in which the directional angle range 31 of the speaker 6 is different in the tilt direction from the angle of view 32 of the first imaging unit 1. In the case of the state shown in FIG. 3(a), not only is it impossible to effectively call out to the person of interest, but there is also a risk that the sound output from the speaker 6 may cause a nuisance to unrelated people other than the person of interest.

[0037] 3(a), that is, when the directional angle range of the speaker 6 is outside the shooting direction of the first imaging unit 1, in other words, when the angle of view and the directional angle range do not overlap, the control unit 8 of this embodiment sets the volume of the speaker 6 to zero. That is, by muting the volume of the speaker 6, the control unit 8 reduces the influence of the sound on unrelated persons other than the person of interest.

[0038] 3(a), the control unit 8 performs tilt control on the third camera head 23 via, for example, the third camera head control unit 12 and the third camera head drive unit 16 so that the audio output direction of the speaker 6 faces the direction of the person of interest. Furthermore, the control unit 8 unmutes the speaker 6 when, as a result of moving the speaker 6 in the tilt direction, part of the directivity angle range 31 of the speaker 6 enters the angle of view of the first imaging unit 1, that is, when part of the directivity angle range 31 overlaps with the angle of view. Note that in the first embodiment, the volume of the speaker 6 after unmuting is set to, for example, the volume before muting.

[0039] In the case of this embodiment, the control unit 8 unmutes the speaker 6 when, for example, a part (one end in this embodiment) of the cone or elliptical cone corresponding to the directional angle range 31 of the speaker 6 comes close to the optical axis of the first imaging unit 1. Here, the control unit 8 can calculate where the one end of the directional angle range 31 of the speaker 6 is located from the audio output direction of the speaker 6 by the third camera platform 23 and the directional angle information of the speaker 6. In addition, the control unit 8 can calculate where the optical axis of the first imaging unit 1 is located from the shooting direction of the first imaging unit 1 by the first camera platform 21 and information on the angle of view of the first imaging unit 1. In the following description of this embodiment, the optical axis of the first imaging unit 1 is referred to as the "center of the angle of view."

[0040] 3(b) shows a state when one end of the directivity angle range 31 of the speaker 6 reaches approximately the center of the angle of view 32 of the first imaging unit 1. Note that when one end of the directivity angle range 31 reaches the center of the angle of view 32 may be determined arbitrarily, such as when one end of the directivity angle range 31 comes into contact with the center of the angle of view 32 or when one end of the directivity angle range 31 falls within a predetermined threshold range from the center of the angle of view 32. In this embodiment, an example will be given in which the speaker 6 is unmuted when one end of the directivity angle range 31 falls within a predetermined threshold range from the center of the angle of view 32.

[0041] Furthermore, when the central axis of the directional angle range of the speaker 6 reaches the center of the angle of view 32 of the first imaging unit 1, the control unit 8 stops the control for changing the orientation of the speaker 6. Note that the control unit 8 can also calculate the central axis of the directional angle range of the speaker 6 from the sound output direction of the speaker 6 by the third camera head 23 and the directional angle information of the speaker 6. In the following description of this embodiment, the central axis of the cone or elliptical cone that corresponds to the directional angle range of the speaker 6 is referred to as the "center of the directional angle range."

[0042] 3(c) shows a state when the center of the directivity angle range 31 of the speaker 6 reaches the center of the angle of view 32 of the first imaging unit 1. The time when the center of the directivity angle range 31 reaches the center of the angle of view 32 may be determined arbitrarily, such as when the center (central axis) of the directivity angle range 31 and the center (optical axis) of the angle of view 32 become substantially parallel in the same direction, or when the relative angle between the central axis and the optical axis falls within a predetermined threshold range. In this embodiment, the control unit 8 stops the control of changing the direction of the speaker 6 when the relative angle between the center (central axis) of the directivity angle range 31 and the center (optical axis) of the angle of view 32 of the first imaging unit 1 falls within a predetermined threshold range.

[0043] Fig. 4 is a flowchart showing the flow of information processing in the control unit 8 of the first embodiment described above. It is assumed that, at the start of the processing of the flowchart in Fig. 4, a person of interest shown in an image captured by the second imaging unit 7 is tracked by the PTZ camera consisting of the first imaging unit 1 and the first pan head 21, and is being called out to by audio output from the speaker 6. Here, an example will be described in which tilt drive control is performed on the speaker 6, as illustrated in Figs. 3(a) to 3(c).

[0044] In step S1, the control unit 8 determines the orientation (shooting direction) of the first imaging unit 1 and the orientation (audio output direction) of the speaker 6. As described above, the control unit 8 determines the shooting direction of the first imaging unit 1 based on the pan amount and tilt amount of the first camera head 21 at the current time, and also determines the orientation (audio output direction) of the speaker 6 based on the tilt amount of the third camera head 23 at the current time.

[0045] Next, in step S2, the control unit 8 determines whether the directional angle range of the speaker 6 is outside the angle of view of the first imaging unit 1 (i.e., whether there is no overlap) based on the information determined in step S1. At this time, the control unit 8 obtains the directional angle range based on the information on the directional angle of the speaker 6 read from the memory 3, and obtains the angle of view from the zoom amount of the first imaging unit 1. Then, the control unit 8 determines whether the directional angle range of the speaker 6 is outside the angle of view of the first imaging unit 1 based on the shooting direction of the first imaging unit 1 and the audio output direction of the speaker 6 determined in step S1, and the information on the directional angle range of the speaker 6 and the angle of view of the first imaging unit 1. If the control unit 8 determines that the directional angle range of the speaker 6 is outside the angle of view of the first imaging unit 1, it proceeds to step S3, whereas if it determines that it is not outside, it returns to step S1 described later.

[0046] In step S3, the control unit 8 transmits control information for muting the speaker 6, that is, control information for setting the volume to zero, to the volume control unit 11. As a result, audio output from the speaker 6 is stopped. Next, in step S4, the control unit 8 controls the speaker 6 to move in the direction of shooting of the first imaging unit 1. At this time, the control unit 8 determines in which direction the speaker 6 should be moved based on the information on the shooting direction of the first imaging unit 1 and the audio output direction of the speaker 6 determined in step S1. The control unit 8 then sends an instruction to the third camera-head control unit 12 to move the speaker 6 in that direction, for example at a constant speed. As a result, the third camera-head control unit 12 controls the third camera-head drive unit 16 to be driven, and the speaker 6 is moved in the tilt direction by the third camera-head 23.

[0047] Next, in step S5, the control unit 8 determines whether or not one end of the directivity angle range of the speaker 6 falls within a predetermined first threshold range from the center of the angle of view of the first imaging unit 1. As an example, in the case of the example shown in Fig. 3(b), the control unit 8 determines whether or not one end of the directivity angle range 31 falls within a predetermined first threshold range from the center of the angle of view 32. If the control unit 8 determines that one end of the directivity angle range of the speaker 6 does not fall within the first threshold range from the center of the angle of view of the first imaging unit 1, the control unit 8 returns the process to step S4, whereas if the control unit 8 determines that one end of the directivity angle range of the speaker 6 falls within the first threshold range, the control unit 8 proceeds to step S6. In step S6, the control unit 8 transmits control information for unmuting the speaker 6, for example, control information for setting the volume to the volume before muting, to the volume control unit 11. As a result, sound is output from the speaker 6.

[0048] Next, in step 7, the control unit 8 calculates the difference between the center of the directivity angle range of the speaker 6 and the center of the angle of view of the first imaging unit 1. For example, the control unit 8 calculates the relative angle between the central axis of the cone or elliptical cone corresponding to the directivity angle range of the speaker 6 and the optical axis of the first imaging unit 1 as the difference between the center of the directivity angle range of the speaker 6 and the angle of view and center of the first imaging unit 1. In other words, the difference between the center of the directivity angle range and the center of the angle of view is information representing the overlapping size between the directivity angle range of the speaker 6 and the angle of view of the first imaging unit 1, or the distance between the center of the directivity angle range of the speaker 6 and the angle of view and center of the first imaging unit 1. In this embodiment, the control unit 8 calculates the difference between the center of the directivity angle range 31 and the center of the angle of view 32 as described in FIG. 3(c).

[0049] Next, in step S8, the control unit 8 determines whether the difference between the center of the directivity angle range of the speaker 6 and the center of the angle of view of the first imaging unit 1 is within a predetermined second threshold range. For example, the control unit 8 determines whether the difference between the center of the directivity angle range and the center of the angle of view is within a predetermined second threshold range. The control unit 8 may also determine whether the difference between the center of the directivity angle range and the center of the angle of view has disappeared. If the control unit 8 determines that the difference is not within the predetermined second threshold range, the control unit 8 proceeds to step S9, whereas if the control unit 8 determines that the difference is within the predetermined second threshold range, the control unit 8 proceeds to step S10.

[0050] Proceeding to step S9, the control unit 8 performs control to move the orientation of the speaker 6 further toward the center of the angle of view of the first imaging unit 1. In the example of FIG. 3(c), the control unit 8 sends an instruction to the third camera-head control unit 12 to move the speaker 6 in the tilt direction toward the center of the angle of view of the first imaging unit 1 at, for example, a constant speed using the third camera-head 23. As a result, the third camera-head control unit 12 performs tilt control to drive the third camera-head driving unit 16, and the speaker 6 is driven in the tilt direction. After this step S9, the control unit 8 returns the process to step S7. On the other hand, if the process proceeds to step S10, the control unit 8 ends the control of changing the orientation of the speaker.

[0051] As described above, in the NW camera of the first embodiment, when tracking of a subject of interest causes the directivity angle range of the speaker 6 to deviate from the angle of view of the first imaging unit 1, the speaker 6 is muted. That is, according to the NW camera of the first embodiment, by performing appropriate audio output control according to the orientations of the first imaging unit 1 and the speaker 6, it is possible to output audio to the subject of interest while reducing the influence of the audio on subjects other than the subject of interest.

[0052] Second Embodiment Next, as a second embodiment, an example in which the volume of the speaker 6 is controlled based on the shooting direction of the first imaging unit 1 and the audio output direction of the speaker 6 will be described. Note that the configuration of the NW camera of the second embodiment is similar to the example shown in Fig. 1, so illustration and description thereof will be omitted. Below, the configuration and processing different from the first embodiment will be mainly described.

[0053] Fig. 5 is a flowchart showing the flow of information processing in the control unit 8 of the second embodiment. In the flowchart of Fig. 5, the same processing steps as those in the flowchart of Fig. 4 are given the same reference numbers as those in Fig. 4, and their explanations will be omitted as appropriate. In Fig. 5, as in the example of Fig. 4, at the start of processing, a person of interest shown in an image captured by the second imaging unit 7 is tracked by a PTZ camera consisting of the first imaging unit 1 and the first pan head 21, and audio is being output from the speaker 6. Also in Fig. 5, an example will be described in which tilt drive control is performed on the speaker 6, as illustrated in Figs. 3(a) to 3(c).

[0054] In the second embodiment, if control unit 8 determines in step S5 that one end of the directivity angle range of speaker 6 falls within the first threshold range from the center of the angle of view of first imaging unit 1, control unit 8 advances the process to step S11. When the process proceeds to step S11, the control unit 8 cancels the mute of the speaker 6 and transmits control information for controlling the volume of the output sound to the volume control unit 11. At this time, the control unit 8 transmits control information for setting the volume of the speaker 6 to, for example, half the volume before the directional angle range of the speaker 6 goes out of the angle of view of the first imaging unit 1 to the volume control unit 11. That is, the volume of the speaker 6 when the mute is cancelled becomes half the volume when the person of interest was being tracked. Note that the half volume is just an example and is not limited to this. After the process of this step S11, the control unit 8 proceeds to the process of step 7.

[0055] In the second embodiment, if it is determined in step S8 that the difference between the center of the directivity angle range of the speaker 6 and the center of the angle of view of the first imaging unit 1 is not within the second threshold range, the control unit 8 proceeds to step S12.

[0056] When the process proceeds to step S12, the control unit 8 sets the difference calculated in step S7 as a reference difference (reference value), and controls the volume of the speaker 6 based on the difference between the reference value and a value recalculated in a later step S13 after the movement control of the speaker in a later step S9. That is, the control unit 8 moves the orientation of the speaker 6 toward the center of the angle of view of the first imaging unit 1 in the later step S9 in the same manner as described above, and further recalculates the difference between the center of the directivity angle range of the speaker 6 after the movement in step S9 and the center of the angle of view of the first imaging unit 1 in the next step S13. Then, after the process of this step S13, the control unit 8 returns the process to step S8.

[0057] Returning from step S13 to the process of step S8, the control unit 8 judges whether the difference recalculated in step S13 (recalculated value) is within the second threshold range or not, and proceeds to step S12 if it is judged that the recalculated value is not within the second threshold range. When proceeding to step S12, the control unit 8 calculates the difference between the reference value obtained in step S7 and the recalculated value already calculated in step S13, and controls the volume of the speaker 6 according to the difference between the reference value and the recalculated value. For example, the control unit 8 sets the volume of the speaker 6 to be louder as the difference between the reference value obtained in step S7 and the recalculated value obtained in step S11 becomes smaller. Here, the difference between the reference value and the recalculated value corresponds to the size of the overlap between the directional angle range of the speaker 6 and the angle of view of the first imaging unit 1, and it can be said that the smaller the difference between the reference value and the recalculated value, the greater the overlap between the directional angle range and the angle of view. That is, the control unit 8 sets the volume of the speaker 6 to be higher as the overlap between the directivity angle range of the speaker 6 and the angle of view of the first imaging unit 1 increases. In step S8, the control unit 8 repeats the processes of steps S12, S9, and S13 until the recalculated value falls within the second threshold range. Then, when the difference between the reference value and the recalculated value falls within the second threshold range, the control unit 8 controls the volume of the speaker 6 to be the volume before muting.

[0058] As described above, in the second embodiment, after the speaker 6 is unmuted, the volume of the speaker 6 is controlled so that the volume increases as the difference between the audio output direction of the speaker 6 and the shooting direction of the first imaging unit 1 decreases. Then, in the second embodiment, when the orientation of the speaker 6 and the orientation of the first imaging unit 1 are roughly aligned, the volume is set to the level before muting. Thus, according to this embodiment, by performing appropriate audio output control according to the orientations of the first imaging unit 1 and the speaker 6, it is possible to output audio to the subject of interest while reducing the impact of the audio on subjects other than the subject of interest.

[0059] <Third embodiment> In the first and second embodiments described above, FIG. 3 is taken as an example, and an example of driving and controlling the speaker 6 in the tilt direction is given. However, it is assumed that the directivity angle range of the speaker 6 does not fall within the angle of view of the first imaging unit 1 by driving only in the tilt direction. Therefore, in the third embodiment, an example of driving and controlling the speaker 6 not only in the tilt direction but also in the pan direction is described. Note that the configuration of the NW camera in the third embodiment is the same as the example shown in FIG. 1, so illustration and description thereof are omitted. In the third embodiment, as in the example of the second embodiment described above, an example of controlling the volume of the speaker 6 based on the difference between the center of the directivity angle range of the speaker 6 and the center of the angle of view of the first imaging unit 1 after the speaker 6 is unmuted is described. Below, a description will be given mainly of configurations and processes different from the second embodiment.

[0060] Fig. 6 is a flowchart showing the flow of information processing in the control unit 8 of the third embodiment. In Fig. 6, similarly to the example of Fig. 5, it is assumed that at the start of processing, a person of interest appearing in an image captured by the second imaging unit 7 is being tracked by the PTZ camera consisting of the first imaging unit 1 and the first pan head 21, and audio is being output from the speaker 6.

[0061] In step S21, the control unit 8 determines the orientations of the first imaging unit 1 and the speaker 6. In the third embodiment, the control unit 8 determines the shooting direction of the first imaging unit 1 based on the tilt amount and pan amount of the first camera head 21 at the current time, and also determines the orientation of the speaker 6 (audio output direction) based on the tilt amount of the third camera head 23 and the pan amount of the fourth camera head 24 at the current time.

[0062] Next, in step S22, the control unit 8 determines whether or not the directional angle range of the speaker 6 has deviated from the angle of view of the first imaging unit 1, based on the orientations of the first imaging unit 1 and the speaker 6 determined in step S1. That is, the control unit 8 determines whether or not the directional angle range has deviated from the angle of view, based on the information on the directional angle of the speaker 6 obtained from the memory 3, the information on the angle of view according to the zoom amount of the first imaging unit 1, and the orientations of the first imaging unit 1 and the speaker 6 obtained in step S21. If the control unit 8 determines that the directional angle range of the speaker 6 has deviated from the angle of view of the first imaging unit 1, it proceeds to step S23, whereas if it determines that the directional angle range has not deviated, it returns to step S21, which will be described later.

[0063] In step S23, the control unit 8 sets the volume of the speaker 6 to be muted, similarly to step S3 described above. Next, in step S24, the control unit 8 performs control to move the orientation of the speaker 6 toward the shooting direction of the first imaging unit 1. In the case of the third embodiment, the control unit 8 determines in which direction the orientation of the speaker 6 is to be moved by a combination of tilt and pan with respect to the shooting direction of the first imaging unit 1. The control unit 8 then sends an instruction to the third tripod head control unit 12 and the fourth tripod head control unit 13 to move the orientation of the speaker 6 in that direction, for example at a constant speed. As a result, the third tripod head control unit 12 and the fourth tripod head control unit 13 perform control to drive the third tripod head driving unit 16 and the fourth tripod head driving unit 17, and the speaker 6 is driven in a direction that is a combination of the tilt direction and the pan direction.

[0064] Next, in step S25, the control unit 8 determines whether or not one end of the directivity angle range of the speaker 6 falls within a first threshold range from the vertical or horizontal center of the angle of view of the first imaging unit 1. If the control unit 8 determines that one end of the directivity angle range of the speaker 6 does not fall within the first threshold range from the vertical or horizontal center of the angle of view of the first imaging unit 1, the control unit 8 returns the process to step S24, whereas if the control unit 8 determines that one end of the directivity angle range of the speaker 6 falls within the first threshold range, the control unit 8 proceeds to step S26.

[0065] 7(a) and 7(b) are diagrams showing a schematic positional relationship between the angle of view 71 of the first imaging unit 1 and the directivity angle range 72 of the speaker 6 in the third embodiment. In FIG. 7(a) and FIG. 7(c), the vertical dashed line in the angle of view 71 represents the vertical center of the angle of view of the first imaging unit 1, and the horizontal dashed line in the angle of view 71 represents the horizontal center of the angle of view of the first imaging unit 1. FIG. 7(a) is a diagram showing an example when one end of the directivity angle range 72 of the speaker 6 enters within the first threshold range from the vertical center of the angle of view 71 of the first imaging unit 1. FIG. 7(b) is a diagram showing an example when one end of the directivity angle range 72 of the speaker 6 enters within the first threshold range from the horizontal center of the angle of view 71 of the first imaging unit 1.

[0066] In step S26, the control unit 8 unmutes the speaker 6 and transmits control information for controlling the volume of the output sound to the volume control unit 11. The volume setting at this time is set to half the volume before the directivity angle range of the speaker 6 goes out of the angle of view of the first imaging unit 1, as in step S11 described above. After processing step S26, the control unit 8 advances the process to step S27.

[0067] In the third embodiment, if it is determined in step S28 that the difference between the center of the directivity angle range of the speaker 6 and the center of the angle of view of the first imaging unit 1 is not within the second threshold range, the process proceeds to step S30. On the other hand, if it is determined that the difference is within the second threshold range, the control unit 8 proceeds to step S29.

[0068] Fig. 7(c), like Fig. 7(a) and Fig. 7(b), is a diagram illustrating a schematic diagram of the positional relationship between the angle of view 71 of the first imaging unit 1 and the directivity angle range 72 of the speaker 6 in the third embodiment. Fig. 7(c) is a diagram illustrating an example when the difference between the center of the directivity angle range 72 of the speaker 6 and the center of the angle of view 71 of the first imaging unit 1 falls within the second threshold range.

[0069] When the process proceeds to step S30, the control unit 8 sets the difference calculated in step S27 as a reference difference (reference value), and controls the volume of the speaker 6 based on the difference between the reference value and a value recalculated in a subsequent step S32 after the speaker movement control in a subsequent step S31. In the case of the third embodiment, the control unit 8 moves the orientation of the speaker 6 toward the center of the angle of view of the first imaging unit 1 by controlling the drive of the third camera head 23 and the fourth camera head 24 in the subsequent step S30. Furthermore, in the next step S32, the control unit 8 recalculates the difference between the center of the directivity angle range of the speaker 6 after the movement in step S31 and the center of the angle of view of the first imaging unit 1. Then, after the process of this step S32, the control unit 8 returns the process to step S28.

[0070] Returning to the process of step S28 from step S32, the control unit 8 judges whether the recalculated value calculated in step S32 is within the second threshold range or not, and proceeds to the process of step S30 if it is judged that the recalculated value is not within the second threshold range. When proceeding to step S30, the control unit 8 calculates the difference between the reference value obtained in step S27 and the recalculated value already obtained in step S32, and controls the volume of the speaker 6 according to the difference between the reference value and the recalculated value. As in the example of the second embodiment described above, the volume control is set so that the smaller the difference between the reference value and the recalculated value, the larger the volume of the speaker 6 is. In step S28, the control unit 8 repeats the processes of steps S30, S31, and S32 until the recalculated value falls within the second threshold range. Then, when the difference between the reference value and the recalculated value falls within the second threshold range, the control unit 8 controls the volume of the speaker 6 to the volume before muting. Thereafter, the process proceeds to step S29, where the control unit 8 ends the control for changing the orientation of the speaker.

[0071] As described above, in the NW camera according to the third embodiment, the orientation of the speaker 6 is moved in the tilt direction and the pan direction, and the volume is controlled to be the volume before muting when the orientation of the speaker 6 and the orientation of the first imaging unit 1 are approximately aligned. The NW camera according to the third embodiment makes it possible to output audio to a subject of interest while reducing the effect of the audio on subjects other than the subject of interest.

[0072] FIG. 8 is a diagram showing a hardware configuration relating to the control unit 8 of the NW camera according to the embodiment described above and its peripheral configuration. 1. The CPU 81 realizes the functions of the image processing unit 2, the first imaging control unit 4, the first camera head control unit 5, the second imaging control unit 9, the second camera head control unit 10, the volume control unit 11, the third camera head control unit 12, and the fourth camera head control unit 13 shown in Fig. 1. The CPU 81 realizes the processes related to each of these functional units by expanding the information processing program according to this embodiment stored in the ROM 82 into the RAM 83 and executing it. The ROM 82 and the RAM 83 realize the functions of the memory 3 shown in Fig. 1.

[0073] The communication circuit 84 is a communication device that realizes the communication unit 18 shown in Fig. 1. The camera I / F 85 is an interface that exchanges image data and control information between the cameras that are the first imaging unit 1 and the second imaging unit 7 shown in Figs. 1 and 2. The speaker I / F 86 is an interface that exchanges audio data and control information with the speaker 6 shown in Figs. 1 and 2. The tripod head I / F 87 is an interface that exchanges pan and tilt control information for the first tripod head drive unit 14, the second tripod head drive unit 15, the third tripod head drive unit 16, and the fourth tripod head drive unit 17 shown in Figs. 1 and 2, and information indicating the results of the tripod head control.

[0074] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main characteristics.

[0075] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) a first control means for controlling a photographing direction of a first imaging device; A second control means for controlling the orientation of the speaker; a volume control means for controlling the volume of the speaker; a setting unit that sets the volume of the speaker controlled by the volume control unit based on the imaging direction of the first imaging device and the orientation of the speaker; 13. An information processing device comprising: (Configuration 2) 2. The information processing device according to configuration 1, wherein the setting means sets the volume of the speaker to be muted when the range of the directivity angle of the speaker does not overlap with the angle of view of the first imaging device. (Configuration 3) The information processing device according to configuration 1, characterized in that the setting means sets the volume of the speaker to be muted when the range of the directional angle of the speaker falls outside the angle of view of the first imaging device due to control of the shooting direction of the first imaging device by the first control means. (Configuration 4) The information processing device according to configuration 2 or 3, wherein the second control means moves the orientation of the speaker toward the shooting direction of the first imaging device after the setting means has muted the volume of the speaker. (Configuration 5) The information processing device according to configuration 4, wherein the setting means sets the mute to be cancelled when a part of the range of the directivity angle of the speaker overlaps with the angle of view of the first imaging device due to control of the orientation of the speaker by the second control means. (Configuration 6) 6. The information processing device according to configuration 5, wherein the setting means sets a volume of the speaker depending on an overlapping degree between a range of a directivity angle of the speaker and an angle of view of the first imaging device. (Configuration 7) 7. The information processing device according to configuration 6, wherein the setting means sets the volume of the speaker to be higher as the overlap between the range of the directivity angle of the speaker and the angle of view of the first imaging device becomes larger. (Configuration 8) The information processing device described in configuration 4, wherein the setting means sets the mute to be released when one end of the range of the directional angle of the speaker falls within a predetermined first threshold range from the optical axis of the first imaging device due to control of the orientation of the speaker by the second control means. (Configuration 9) The information processing device according to configuration 8, wherein the setting means sets the volume of the speaker based on a difference between an angle between a central axis of a range of a directivity angle of the speaker when the mute is released and the optical axis of the first imaging device, and an angle between a central axis of a range of a directivity angle of the speaker after the second control means has controlled the orientation of the speaker and the optical axis of the first imaging device. (Configuration 10) 10. The information processing device according to configuration 9, wherein the setting means sets the volume of the speaker to be larger as the difference in the angles becomes smaller. (Configuration 11) The information processing device according to configuration 10, wherein the setting means sets the volume of the speaker when the mute is released to a volume lower than the volume before the muting, and when the difference in the angle falls within a predetermined second threshold range, sets the volume of the speaker to the volume before the muting. (Configuration 12) The speaker is mounted on a tilt head that can be driven in a tilt direction, and the tilt head is further mounted on a pan head that can be driven in a pan direction, The information processing device described in any one of configurations 1 to 11, characterized in that the second control means controls the orientation of the speaker by at least one of driving control in a tilt direction for the tilt head and driving control in a pan direction for the pan head. (Configuration 13) a third control unit for controlling a photographing direction of a second image pickup device different from the first image pickup device; the first imaging device is mounted on a first camera platform that is movable in a pan direction and a tilt direction; the second imaging device is mounted on a second camera platform that is movable in a tilt direction; The second camera head on which the second imaging device is mounted is further mounted on the pan head, The information processing device described in configuration 12, characterized in that the third control means controls the shooting direction of the third imaging device by at least one of driving control in a tilt direction for the second camera head and driving control in a pan direction for the pan camera head. (Configuration 14) the second imaging device is an imaging device having a wider angle of view than the first imaging device, The information processing device according to configuration 13, wherein the first control means controls the shooting direction of the first imaging device so that a subject of interest appearing in an image captured by the second imaging device is tracked by the first imaging device. (Method 1) a first control step of controlling an imaging direction of a first imaging device; a second control step of controlling the orientation of the speaker; a volume control step of controlling the volume of the speaker; a setting step of setting a volume of the speaker controlled in the volume control step based on an imaging direction of the first imaging device and an audio output direction of the speaker; 13. An information processing method comprising: (Program 1) A program for causing a computer to function as the information processing device according to any one of configurations 1 to 13. [Explanation of symbols]

[0076] 1: first imaging unit, 2: image processing unit, 3: memory, 4: first imaging control unit, 5: first pan head control unit, 6: speaker, 7: second imaging unit, 8: control unit, 9: second imaging control unit, 10: second pan head control unit, 11: volume control unit, 12: third pan head control unit, 13: fourth pan head control unit, 14: first pan head drive unit, 15: second pan head drive unit, 16: third pan head drive unit, 17: fourth pan head drive unit

Claims

1. a first control means for controlling a photographing direction of a first imaging device; A second control means for controlling the orientation of the speaker; a volume control means for controlling the volume of the speaker; a setting unit that sets the volume of the speaker controlled by the volume control unit based on the imaging direction of the first imaging device and the orientation of the speaker; 13. An information processing device comprising:

2. 2 . The information processing apparatus according to claim 1 , wherein the setting unit sets the volume of the speaker to be muted when the range of the directivity angle of the speaker does not overlap with the angle of view of the first imaging device.

3. The information processing device according to claim 1, characterized in that the setting means sets the volume of the speaker to be muted when the range of the directional angle of the speaker falls outside the angle of view of the first imaging device due to control of the shooting direction of the first imaging device by the first control means.

4. 4. The information processing device according to claim 2, wherein the second control means moves the orientation of the speaker toward the imaging direction of the first imaging device after the setting means has muted the volume of the speaker.

5. The information processing device according to claim 4, characterized in that the setting means sets the mute to be cancelled when a part of the range of the directivity angle of the speaker overlaps with the angle of view of the first imaging device due to control of the orientation of the speaker by the second control means.

6. 6. The information processing apparatus according to claim 5, wherein the setting means sets the volume of the speaker in accordance with the extent of overlap between a range of a directivity angle of the speaker and an angle of view of the first imaging device.

7. 7. The information processing apparatus according to claim 6, wherein the setting means sets the volume of the speaker to be larger as the overlap between the range of the directivity angle of the speaker and the angle of view of the first imaging device becomes larger.

8. The information processing device according to claim 4, characterized in that the setting means sets the mute to be released when one end of the range of the directional angle of the speaker falls within a predetermined first threshold range from the optical axis of the first imaging device due to control of the orientation of the speaker by the second control means.

9. The information processing device according to claim 8, characterized in that the setting means sets the volume of the speaker based on a difference between an angle between a central axis of a range of a directivity angle of the speaker when the mute is released and the optical axis of the first imaging device, and an angle between a central axis of a range of a directivity angle of the speaker after control of the orientation of the speaker by the second control means and the optical axis of the first imaging device.

10. 10. The information processing apparatus according to claim 9, wherein the setting means sets the volume of the speaker to be larger as the difference in the angles becomes smaller.

11. The information processing device according to claim 10, characterized in that the setting means sets the volume of the speaker when the mute is released to a volume lower than the volume before the muting, and when the difference in the angle falls within a predetermined second threshold range, sets the volume of the speaker to the volume before the muting.

12. The speaker is mounted on a tilt head that can be driven in a tilt direction, and the tilt head is further mounted on a pan head that can be driven in a pan direction, 2. The information processing apparatus according to claim 1, wherein the second control means controls the orientation of the speaker by at least one of driving control of the tilt head in a tilt direction and driving control of the pan head in a pan direction.

13. a third control unit for controlling a photographing direction of a second image pickup device different from the first image pickup device; the first imaging device is mounted on a first camera platform that is movable in a pan direction and a tilt direction; the second imaging device is mounted on a second camera platform that is movable in a tilt direction; The second camera head on which the second imaging device is mounted is further mounted on the pan head, The information processing device according to claim 12, characterized in that the third control means controls the shooting direction of the third imaging device by at least one of driving control in a tilt direction for the second camera head and driving control in a pan direction for the pan camera head.

14. the second imaging device is an imaging device having a wider angle of view than the first imaging device, The information processing device according to claim 13 , wherein the first control means controls a shooting direction of the first imaging device so that a subject of interest appearing in an image captured by the second imaging device is tracked by the first imaging device.

15. a first control step of controlling an imaging direction of a first imaging device; a second control step of controlling the orientation of the speaker; a volume control step of controlling the volume of the speaker; a setting step of setting a volume of the speaker controlled in the volume control step based on an imaging direction of the first imaging device and an audio output direction of the speaker; 13. An information processing method comprising:

16. Computer, a first control means for controlling a photographing direction of a first imaging device; A second control means for controlling the orientation of the speaker; a volume control means for controlling the volume of the speaker; a setting unit that sets the volume of the speaker controlled by the volume control unit based on an imaging direction of the first imaging device and an audio output direction of the speaker; A program that causes the device to function as an information processing device having the above-mentioned configuration.

Images