Information processing apparatus and information processing method

Abstract

To control an imaging direction of an imaging device in a direction of a sound source even in a case where arrangement of a sound detection device and an imaging device is changed.SOLUTION: An information processing apparatus 24 comprises: a first calculation unit 242 that calculates a coordinate of a person 30 using a position of an imaging device 20 as a reference on the basis of a direction of the imaging device 20 which exists in the same space as a sound detection device 10 using a position of the sound detection device 10 as a reference being detected by the sound detection device 10 and a direction of a person 30 who exists in the same space as the sound detection device 10 using a position of the sound detection device 10 as a reference being detected by the sound detection device 10; and a second calculation unit 243 that calculates a horizontal direction rotation angle and a vertical direction rotation angle of the imaging device 20 on the basis of a coordinate of the person 30 using a position of the imaging device 20 as a reference.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Patent document 1 describes an image monitoring device that identifies the direction of a sound source by analyzing sound data detected by multiple microphones arranged in different directions in a horizontal plane, and automatically controls the imaging direction of a video camera to the identified direction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-344957 Summary of the Invention [Problem to be solved by the invention]

[0004] In the image monitoring device described in Patent Document 1, multiple microphones are arranged evenly at approximately equal angles to the left and right of the horizontal plane, with the same direction as the center of the horizontal direction of the area monitored by the video camera as the center. That is, in the image monitoring device described in Patent Document 1, in order to control the shooting direction of the video camera toward the sound source, the microphones and the video camera must be arranged in a predetermined position. Therefore, if the positions of the microphones and the video camera are changed, it becomes impossible to control the shooting direction of the video camera toward the sound source.

[0005] The present invention aims to provide an information processing device and an information processing method that can control the shooting direction of the imaging device to the direction of a sound source even when the positions of the sound detection device and the imaging device change. [Means for solving the problem]

[0006] The present invention provides an information processing device including: a first calculation unit that calculates coordinates of a person based on the position of an imaging device, based on the direction of the imaging device that is present in the same space as the voice detection device and that is detected by the voice detection device and that is based on the position of the voice detection device; and a second calculation unit that calculates horizontal and vertical rotation angles of the imaging device, based on the coordinates of the person based on the position of the imaging device.

[0007] The present invention provides an information processing method for calculating the coordinates of a person based on the position of an imaging device, based on the direction of an imaging device that is present in the same space as the voice detection device and that is detected by the voice detection device and that is based on the position of the voice detection device, and for calculating the horizontal rotation angle and vertical rotation angle of the imaging device based on the calculated coordinates of the person based on the position of the imaging device. [Effects of the Invention]

[0008] According to the information processing device and information processing method of the present invention, even if the positions of the sound detection device and the imaging device change, the imaging direction of the imaging device can be controlled to the direction of the sound source. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a camera control system including an information processing device according to an embodiment. [Figure 2] FIG. 2 is a diagram showing three-dimensional polar coordinates θ and φ with the voice detection device as the reference. [Figure 3] FIG. 3 is a diagram showing three-dimensional polar coordinates φ with the voice detection device as the reference. [Figure 4] FIG. 4 is a diagram showing the directions of the x-axis, y-axis, and z-axis with the voice detection device as the reference. [Figure 5]FIG. 5 is a diagram showing the directions of the x-axis, y-axis, and z-axis with the imaging device as the reference. [Figure 6] FIG. 6 is a flowchart showing an example of the operation of the voice detection device. [Figure 7] FIG. 7 is a diagram showing three-dimensional polar coordinates indicating the direction of the image capture device 20 with reference to the position of the voice detection device. [Figure 8] FIG. 8 is a diagram showing three-dimensional polar coordinates indicating the direction of a person with respect to the position of the voice detection device. [Figure 9] FIG. 9 is a flowchart showing an example of the operation of the information processing device according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing a three-dimensional orthogonal coordinate system indicating the position of the imaging device relative to the position of the sound detection device. [Figure 11] FIG. 11 is a diagram showing three-dimensional orthogonal coordinates indicating the position of a person with respect to the position of the voice detection device. [Figure 12] FIG. 12 is a diagram showing the pan angle of the imaging device. [Figure 13] FIG. 13 is a diagram showing the tilt angle of the imaging device. [Figure 14] FIG. 14 is a flowchart showing an example of the operation of the information processing device according to the second embodiment. [Figure 15] FIG. 15 is a diagram showing an example of a GUI for inputting the vertical arrangement of the audio detection device and the imaging device. [Figure 16] FIG. 16 is a diagram showing an example of a GUI for inputting the horizontal arrangement of the audio detection device and the imaging device. DETAILED DESCRIPTION OF THE INVENTION

[0010] An information processing apparatus according to an embodiment will be described below with reference to the accompanying drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0011] [First embodiment] Referring to FIG. 1, an example of the configuration of a camera control system including an information processing device (control unit) 24 according to the first embodiment will be described. The camera control system is mainly composed of a ceiling microphone 10, which is an audio detection device, and a pan-tilt-zoom camera 20, which is an imaging device. Hereinafter, the pan-tilt-zoom camera 20 will be abbreviated as a PTZ camera 20. The ceiling microphone 10 and the PTZ camera 20 are placed in the same room, such as a conference room. That is, the ceiling microphone 10 and the PTZ camera 20 exist in the same space. The ceiling microphone 10 is installed on the ceiling, for example. The PTZ camera 20 is installed at the front of the room, for example. The ceiling microphone 10 and the PTZ camera 20 are connected by wire or wirelessly. The PTZ camera 20 is connected to an external monitor 40 by wire or wirelessly.

[0012] The ceiling microphone 10 includes a sound detection unit 11, an angle calculation unit 12, and a transmission unit 13. The sound detection unit 11 is a microphone that detects sounds generated from a sound source within the target range. For example, the sound detection unit 11 detects a predetermined sound emitted from a PTZ camera 20 within the target range. The sound detection unit 11 also detects sounds emitted from a person within the target range. The sound detection unit 11 generates an electrical signal corresponding to the detected sound.

[0013] The angle calculation unit 12 detects the direction of the sound source relative to the position of the ceiling microphone 10, based on the electrical signal corresponding to the sound detected by the voice detection unit 11. The angle calculation unit 12 calculates, as the direction of the sound source, three-dimensional polar coordinates θ and φ of the sound source relative to the position of the ceiling microphone 10, as shown in Figures 2 and 3, for example, based on the electrical signal corresponding to the sound detected by the voice detection unit 11. The three-dimensional polar coordinates θ and φ are two angle components that indicate the direction of the sound source relative to the position of the ceiling microphone 10.

[0014] As shown in FIG. 2, the three-dimensional polar coordinate θ of the sound source s, based on the position of the ceiling microphone 10, is the angle between the z-axis, which is drawn vertically from the center of the ceiling microphone 10 installed on the ceiling of the room, and the line connecting the center of the ceiling microphone 10 and the sound source s. FIG. 3 is a view looking down on the x-y plane on which the sound source s exists, from a perspective where the ceiling microphone 10 in FIG. 2 is viewed from the back. Note that the position of the sound source s in FIG. 2 is different from the position of the sound source s in FIG. 3. As shown in FIG. 3, the three-dimensional polar coordinate φ of the sound source s, based on the position of the ceiling microphone 10, is the angle between the x-axis and the line connecting the intersection O of the z-axis, which is drawn vertically from the center of the ceiling microphone 10 installed on the ceiling, and the x-y plane on which the sound source s exists, when the intersection O is set as the origin. The angular range that the three-dimensional polar coordinate φ can take is the counterclockwise angular range of 0 to 180° from the first quadrant to the second quadrant on the xy plane shown in Figure 3, and the clockwise angular range of 0° to -180° from the fourth quadrant to the third quadrant.

[0015] Furthermore, the angle calculation unit 12 calculates the distance d from the ceiling microphone 10 to the sound source s based on the amplitude of the electrical signal corresponding to the sound detected by the sound detection unit 11 or a sensor signal from a distance sensor such as a TOF (Time Of Flight) sensor (not shown). Note that the distance d from the ceiling microphone 10 to the sound source s may be a value input by the user. The three-dimensional polar coordinate d is a distance component that indicates the distance to the sound source s based on the position of the ceiling microphone 10. Details of a method for calculating or acquiring the distance d from the ceiling microphone 10 to the sound source s will be described later with reference to FIG. 6.

[0016] The transmitter 13 transmits to the PTZ camera 20 the position information of the sound source s detected by the angle calculation unit 12, based on the position of the ceiling microphone 10, i.e., the three-dimensional polar coordinates (d, θ, φ) of the sound source s based on the position of the ceiling microphone 10.

[0017] Here, in the first embodiment, as shown in Figures 4 and 5, the x-axis direction and z-axis direction set as the reference for the ceiling microphone 10 are the same as the x-axis direction and z-axis direction set as the reference for the PTZ camera 20, and the PTZ camera 20 is installed on a desk rather than being suspended from the ceiling.

[0018] The PTZ camera 20 includes a receiving unit 21 , a storage unit 22 , an audio output unit 23 , a control unit 24 , a pan driving unit 25 , a tilt driving unit 26 , and an imaging unit 27 .

[0019] The receiving unit 21 receives, from the ceiling microphone 10, position information of the sound source s relative to the position of the ceiling microphone 10, i.e., the three-dimensional polar coordinates (d, θ, φ) of the sound source s relative to the position of the ceiling microphone 10. The receiving unit 21 may also receive an instruction signal from a remote controller (not shown) or the like, or may receive the instruction signal via a network.

[0020] The memory unit 22 stores various information received from the ceiling microphone 10. The audio output unit 23 outputs a predetermined sound when the receiving unit 21 receives an instruction signal from a remote controller (not shown) or when an input is received from a button (not shown) or within a predetermined time after the camera control system is started. The audio output unit 23 outputs the predetermined sound at a preset sound pressure level. Here, the predetermined sound is preferably a sound in a frequency band different from the voice that a person can make, and may be a continuous sound of a certain frequency such as a "beep beep," an intermittent buzzer, or a beep. Furthermore, any sound other than the audible band may be detectable by the audio detection unit 11.

[0021] The control unit 24 is, for example, a general-purpose computer equipped with a CPU (Central Processing Unit), memory, input / output units, etc. A computer program for causing the computer to function as the control unit 24 is installed in the computer. The computer realizes multiple functions by executing the computer program.

[0022] The control unit 24 has, as a plurality of functions, a coordinate conversion unit 241, a relative coordinate calculation unit 242, a rotation angle calculation unit 243, and a direction control unit 244. Details of each function will be described later.

[0023] The pan driving unit 25 rotates the imaging unit 27 in a horizontal plane based on a control signal from the control unit 24. The tilt driving unit 26 rotates the imaging unit 27 in a vertical plane based on a control signal from the control unit 24.

[0024] The imaging unit 27 includes a lens and an imaging element, and functions to capture an image of a subject. The imaging element is a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor. The imaging unit 27 generates and outputs a video signal for each frame based on an imaging signal supplied from the imaging element. The video signal generated by the imaging unit 27 may be displayed on an external monitor 40, recorded on a recording medium, or transmitted to an external device via a network.

[0025] Next, an example of the flow of operations of the camera control system will be described with reference to Figs. 6 to 13. Fig. 6 shows an example of the operation of the ceiling microphone 10. First, in step S101 of Fig. 6, the sound detection unit 11 of the ceiling microphone 10 detects a predetermined sound emitted from the PTZ camera 20 within the target range. The sound detection unit 11 generates an electrical signal corresponding to the detected sound. The angle calculation unit 12 detects the direction of the PTZ camera 20 based on the position of the ceiling microphone 10, based on the electrical signal corresponding to the detected sound. Specifically, as shown in Fig. 7, the angle calculation unit 12 calculates, as the direction of the PTZ camera 20, angle information θ cam , and φ cam Calculate.

[0026] Furthermore, the angle calculation unit 12 calculates the distance d from the ceiling microphone 10 to the PTZ camera 20 based on the amplitude of the electrical signal corresponding to the detected sound and a conversion table stored in advance for converting the amplitude of the electrical signal into the distance to the PTZ camera 20. cam That is, the angle calculation unit 12 calculates the distance information d cam The conversion table may be determined in advance by experiment or simulation. The angle calculation unit 12 calculates the distance d from the ceiling microphone 10 to the PTZ camera 20 based on a sensor signal from a distance sensor such as a TOF sensor (not shown). cam may be calculated.

[0027] Proceeding to step S102, the transmitting unit 13 transmits the position information of the PTZ camera 20 based on the position of the ceiling microphone 10 to the PTZ camera 20. That is, the transmitting unit 13 transmits the three-dimensional polar coordinates (d cam ,θ cam ,φ cam ) to the PTZ camera 20. At this time, as a method of indicating that the transmitted three-dimensional polar coordinates are position information of the PTZ camera 20, the transmitter 13 may transmit identification information indicating that the position information of the PTZ camera 20 is the three-dimensional polar coordinates together with the three-dimensional polar coordinates, or may predetermine that the three-dimensional polar coordinates transmitted first by the PTZ camera 20 after the PTZ camera 20 outputs a predetermined sound within a predetermined time after the camera control system is started are the position information of the PTZ camera 20.

[0028] The process proceeds to step S103, and if the operation is to be ended by a user instruction (YES in step S103), the ceiling microphone 10 ends the operation of Fig. 6. On the other hand, if the operation is not to be ended by a user instruction (NO in step S103), the process proceeds to step S104.

[0029] In step S104, if the voice detection unit 11 of the ceiling microphone 10 has not detected a voice emitted from a person in the target range (NO in step S104), the process returns to step S103. On the other hand, if the voice detection unit 11 of the ceiling microphone 10 has detected a voice emitted from a person in the target range (YES in step S104), the process proceeds to step S105.

[0030] In step S105, the angle calculation unit 12 detects the direction of the person based on the position of the ceiling microphone 10, based on the electrical signal corresponding to the detected sound. Specifically, as shown in Fig. 8, the angle calculation unit 12 calculates, as the direction of the person, θ man , and φ man Calculate.

[0031] The angle calculation unit 12 also calculates the distance d from the ceiling microphone 10 to the person 30 based on a sensor signal from a distance sensor such as a TOF sensor (not shown). man That is, the angle calculation unit 12 calculates distance information d man The distance d from the ceiling microphone 10 to the person 30 is calculated. man may use a value input by the user.

[0032] Thereafter, the process proceeds to step S106, where the transmitter 13 transmits the position information of the person 30 relative to the position of the ceiling microphone 10 to the PTZ camera 20. That is, the transmitter 13 transmits the three-dimensional polar coordinates (d man ,θ man ,φ man) to the PTZ camera 20. At this time, as a method of indicating that the transmitted three-dimensional polar coordinates are position information of the person 30, the transmitter 13 may transmit identification information indicating that they are position information of the person 30 together with the three-dimensional polar coordinates, or may predetermine that the second or subsequent three-dimensional polar coordinates to be transmitted after the PTZ camera 20 outputs a predetermined sound within a predetermined time after the camera control system is started are position information of the person 30. Then, the process returns to step S103.

[0033] 9 shows an example of the operation of the PTZ camera 20 according to the first embodiment. In step S201 of FIG. 9, if the receiving unit 21 of the PTZ camera 20 has not received position information from the ceiling microphone 10 (NO in step S201), the operation returns to step S201. On the other hand, if the receiving unit 21 has received position information from the ceiling microphone 10 (YES in step S201) in step S201, the operation proceeds to step S202.

[0034] In step S202, the receiving unit 21 determines whether the position information received from the ceiling microphone 10 is position information of the PTZ camera 20 or position information of the person 30, based on the identification information. Alternatively, it may be determined that the first three-dimensional polar coordinates received after the camera control system is started are position information of the PTZ camera 20, and the second or subsequent three-dimensional polar coordinates received are position information of the person 30. If the received position information is position information of the PTZ camera 20 (YES in step S202), the process proceeds to step S203. On the other hand, if the received position information is not position information of the PTZ camera 20 (NO in step S202), the process proceeds to step S204.

[0035] In step S203, the receiving unit 21 receives the position information of the PTZ camera 20, that is, the three-dimensional polar coordinates (d cam ,θ cam ,φ cam) is stored and held in the storage unit 22 as the first camera position. The coordinate conversion unit 241 converts the three-dimensional polar coordinates (d cam ,θ cam ,φ cam ), a three-dimensional Cartesian coordinate (x) indicating the position of the PTZ camera 20 relative to the center position of the ceiling microphone 10 is calculated. cam ,y cam ,z cam ) is calculated.

[0036] Here, h in Figure 10 cam and r cam can be calculated using equations (1) and (2), respectively.

number

number

number

[0037] y cam and z cam can be calculated in the same way using equations (4) and (5), respectively.

number

number

[0038] Then, the coordinate conversion unit 241 converts the calculated three-dimensional orthogonal coordinates (x cam ,y cam ,z cam ) is stored in the storage unit 42 as the second camera position, and the operation proceeds to step S207.

[0039] In step S204, the receiving unit 21 receives the position information of the person 30, that is, the three-dimensional polar coordinates (d man ,θ man ,φ man ) is stored and held in the storage unit 22 as the first person position. Then, the coordinate conversion unit 241 converts the three-dimensional polar coordinates (d man ,θ man ,φ man ), three-dimensional Cartesian coordinates (x man ,y man ,z man ) is calculated.

[0040] Here, the three-dimensional orthogonal coordinates (x man ,y man ,z man ) are three-dimensional Cartesian coordinates (x cam ,y cam ,z cam ), it can be calculated using the following equations (6) to (8).

number

number

number

[0041] Then, the coordinate conversion unit 241 converts the calculated three-dimensional orthogonal coordinates (x man ,y man ,z man ) is stored in the storage unit 42 as the second person position, and the operation proceeds to step S205.

[0042] In step S205, the relative coordinate calculation unit 242 calculates the position of the person 30 relative to the position of the PTZ camera 20, based on the second person position calculated in step S204 and the second camera position stored in the storage unit 42. Specifically, the relative coordinate calculation unit 242 calculates the position of the person 30 as a three-dimensional orthogonal coordinate (x man ,y man ,z man ) and three-dimensional Cartesian coordinates (x cam ,y cam ,z cam ) and calculate three-dimensional Cartesian coordinates (x, y, z) that indicate the position of the person 30 relative to the position of the PTZ camera 20. tra ,y tra ,z tra ) is calculated.

[0043] Here, the three-dimensional orthogonal coordinates (x tra ,y tra ,z tra ) can be calculated using the following equations (9) to (11).

number

number

number

[0044] The operation then proceeds to step S206, where the rotation angle calculation unit 243 calculates the pan angle, which is the horizontal rotation angle of the PTZ camera 20, and the tilt angle, which is the vertical rotation angle, based on the position of the person 30 relative to the position of the PTZ camera 20 calculated in step S205.

[0045] Here, the pan angle, which is the horizontal rotation angle of the PTZ camera 20, is an angle φ between the x-axis on the xy plane with the position of the PTZ camera 20 as the reference (origin) and a line connecting the positions of the PTZ camera 20 and the person 30, as shown in FIG. traIn the PTZ camera 20 of this embodiment, the pan angle φ tra The angle range that can be taken by the PTZ camera 20 is set to be the counterclockwise angle range of 0° to 175° from the first quadrant to the second quadrant on the xy plane shown in Fig. 12, and the clockwise angle range of 0° to -175° from the fourth quadrant to the third quadrant. Therefore, the pan angle φ of the PTZ camera 20 is set to be the counterclockwise angle range of 0° to -175° from the fourth quadrant to the third quadrant on the xy plane shown in Fig. 12. tra When calculating, the possible angle range is -175°≦φ tra ≦175°. Of course, these angle ranges are set according to the specifications of the PTZ camera in this embodiment, and the present invention is not limited to the angle ranges described here. The same applies to the angle ranges in the following description.

[0046] The pan angle φ in Fig. 12 tra is (x tra ,y tra ) but x tra >0 and y tra If the condition of =0 is satisfied, it can be calculated using the following equation (12).

number

[0047] The pan angle φ in Fig. 12 tra is (x tra ,y tra ) but x tra >0 and y tra >0, x tra <0 and y tra >0, x tra >0 and y tra <0, x tra <0 and y tra <0, it can be calculated using the following formula (13).

number

[0048] The pan angle φ in Fig. 12 tra is (x tra ,ytra ) but x tra =0 and y tra If the condition of >0 is met, it can be calculated using the following formula (14).

number

[0049] The pan angle φ in Fig. 12 tra is (x tra ,y tra ) but x tra =0 and y tra If the condition <0 is met, it can be calculated using the following formula (15).

number

[0050] The pan angle φ in Fig. 12 tra is (x tra ,y tra ) but x tra <0 and y tra If the condition of =0 is satisfied, it can be calculated using the following equation (16) based on the range of angles that can be taken.

number

[0051] Next, the tilt angle, which is the vertical rotation angle of the PTZ camera 20, is determined by the angle θ between the x-axis on the xz plane, which has the position of the PTZ camera 20 as the reference (origin), and the line connecting the positions of the PTZ camera 20 and the person 30, as shown in FIG. tra The tilt angle θ tra The angular range that can be taken by is the counterclockwise angular range of 0° to 90° in the first quadrant on the xz plane shown in FIG. 13, and the clockwise angular range of 0° to −30° in the fourth quadrant.

[0052] Tilt angle θ in Figure 13 tra can be calculated using the following equation (17).

number

[0053] Proceeding to step S207, the direction control unit 244 adjusts the pan angle φ calculated in step S206. tra The direction control unit 244 controls the pan driving unit 25 based on the tilt angle θ calculated in step S206. tra The tilt drive unit 26 is controlled based on the above.

[0054] Thereafter, the operation proceeds to step S208, and if the operation is to be ended by the user instruction (YES in step S208), the PTZ camera 20 ends the operation of Fig. 9. On the other hand, if the operation is not to be ended by the user instruction (NO in step S208), the operation returns to step S201.

[0055] (Operation and effect of the first embodiment) As described above, the information processing device (control unit) 24 according to the first embodiment includes a first calculation unit (relative coordinate calculation unit) 242 and a second calculation unit (rotation angle calculation unit) 243. The first calculation unit 242 calculates the direction θ of the imaging device (PTZ camera) 20 present in the same space as the voice detection device 10, based on the position of the voice detection device 10, detected by the voice detection device (ceiling microphone) 10. cam ,φ cam and a direction θ of a person 30 present in the same space as the voice detection device 10, based on the position of the voice detection device 10, detected by the voice detection device 10. man ,φ man Based on this, the coordinates (x tra ,y tra ,z tra The second calculation unit (rotation angle calculation unit) 243 calculates the coordinates (x tra ,y tra ,z tra ), the horizontal rotation angle φ of the image pickup device 20 is calculated based on the tra and the vertical rotation angle θ tra Calculate.

[0056] The imaging device 20 according to the first embodiment also includes a pan driver 25 and a tilt driver 26, and the information processing device 24 further includes a direction controller 244. The direction controller 244 calculates the horizontal rotation angle φ of the imaging device calculated by the second calculator 243. tra Based on this, the pan driving unit 25 of the image capturing device 20 is controlled to rotate the image capturing device 20 in the vertical direction at a rotation angle θ tra The tilt drive unit 26 is controlled based on the above.

[0057] The information processing device 24 calculates the direction θ of the image capturing device 20 based on the position of the voice detection device 10. cam ,φ cam and the direction θ of the person 30 relative to the position of the voice detection device 10. man ,φ man Based on this, the coordinates (x tra ,y tra ,z tra The information processing device 24 can calculate the coordinates (x tra ,y tra ,z tra ), the horizontal rotation angle φ of the image capturing device 20 is calculated based on the tra and the vertical rotation angle θ tra This makes it possible to control the imaging direction of the imaging device 20 to the direction of the sound source even if the positions of the sound detection device 10 and the imaging device 20 change.

[0058] Furthermore, the first calculation unit 242 of the information processing device 24 according to the first embodiment calculates the distance d from the voice detection device 10 to the image capture device 20 and the person 30, which is detected based on the amplitude of the sound signal detected by the voice detection device 10 or the sensor signal of the distance sensor mounted on the voice detection device 10. cam , d man and the direction θ of the image capture device 20 relative to the position of the voice detection device 10. cam ,φ cam and the direction θ of the person 30 relative to the position of the voice detection device 10. man ,φ man Based on this, the coordinates (xtra ,y tra , z tra ) is calculated.

[0059] The information processing device 24 calculates the distance d to the imaging device 20 based on the position of the voice detection device 10. cam and the direction θ of the image capture device 20 relative to the position of the voice detection device 10. cam ,φ cam Based on this, the position coordinates (x cam ,y cam ,z cam The information processing device 24 can calculate the distance d to the person based on the position of the voice detection device 10. man and the direction θ of the person 30 relative to the position of the voice detection device 10. man ,φ man Based on this, the position coordinates (x man ,y man ,z man The information processing device 24 can calculate the position coordinates (x cam ,y cam ,z cam ) and the position coordinates (x man ,y man ,z man ) based on the position coordinates (x tra ,y tra ,z tra ) can be calculated. This allows the imaging direction of the imaging device 20 to be accurately controlled to the direction of the sound source.

[0060] [Second embodiment] In the first embodiment, as shown in FIGS. 4 and 5 , the x-axis and z-axis directions set as references for the ceiling microphone 10 are the same as the x-axis and z-axis directions set as references for the PTZ camera 20, and the PTZ camera 20 is installed on a desk rather than a ceiling-mounted camera. In contrast, the second embodiment of the present invention will be described, focusing on differences from the first embodiment, regarding the operation of the information processing device 24 when at least one of the x-axis and z-axis directions set as references for the ceiling microphone 10 and the x-axis and z-axis directions set as references for the PTZ camera 20 is not the same (case of axis misalignment). In the second embodiment, the PTZ camera 20 may be a ceiling-mounted camera. Furthermore, the PTZ camera 20 in the second embodiment cannot be rotated more than 180° in one direction. Furthermore, the possible angle range of the pan angle and tilt angle of the PTZ camera 20 is −175°≦φ, similar to the PTZ camera 20 in the first embodiment. tra It is assumed that the angle is set to ≦175°.

[0061] Fig. 14 shows an example of the operation of the PTZ camera 20 according to the second embodiment of the present invention. The operations of steps S302 to S307 and steps S309 to S310 in Fig. 14 are the same as the operations of steps S201 to S208 in Fig. 9, and therefore description thereof will be omitted.

[0062] First, in step S301 of Fig. 14, the PTZ camera 20 receives input from the user regarding the placement of the ceiling microphone 10 and the PTZ camera 20. For example, the control unit 24 displays a GUI (Graphical User Interface) for inputting the placement of the ceiling microphone 10 and the PTZ camera 20 on an operation display unit (not shown) or the like, and the user selects the GUI to receive input regarding the placement of the ceiling microphone 10 and the PTZ camera 20. Details of the GUI for inputting the placement of the ceiling microphone 10 and the PTZ camera 20 will be described later with reference to Figs. 15 and 16. Note that the PTZ camera 20 may automatically determine whether it is a ceiling-suspended PTZ camera using a gyro sensor (not shown) or the like.

[0063] In step S301, after receiving an input from the user regarding the placement of the ceiling microphone 10 and the PTZ camera 20, the process proceeds to step S302, where the PTZ camera 20 performs the same operations as in steps S201 to S206 in FIG. 9 in steps S302 to S307.

[0064] Thereafter, the process proceeds to step S308, and the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation of the PTZ camera 20 with respect to the ceiling microphone 10 due to the arrangement of the PTZ camera 20 with respect to the ceiling microphone 10, which was received in step S301. tra and tilt angle θ tra Calculate.

[0065] Here, referring to Figures 15 and 16, we will explain an example of a GUI for inputting the placement of the ceiling microphone 10 and the PTZ camera 20, and an example of a method for calculating the pan angle and tilt angle according to the axial misalignment between the ceiling microphone 10 and the PTZ camera 20.

[0066] For example, the left diagram of Fig. 15 shows a case where the PTZ camera 20 has a cylindrical shape and is placed on a desk facing upward. In a scene like the one shown in the left diagram of Fig. 15, the rotation angle calculation unit 243 calculates the pan angle φ tra and tilt angle θ tra is used as is.

[0067] 15 shows a case where the PTZ camera 20 has a cylindrical shape, is suspended from the ceiling, and is installed facing downward. In a scene such as that shown in FIG. 15, the rotation angle calculation unit 243 calculates the tilt angle θ tra′ =-θ tra Calculate.

[0068] 16, a rectangle indicates the ceiling microphone 10, a circle indicates the PTZ camera 20, and the direction of the arrow indicates the x-axis direction of each of the ceiling microphone 10 and the PTZ camera 20. The rotation angle calculation unit 243 calculates a pan angle so that the PTZ camera 20 cannot rotate by 180° or more in one direction, and if the rotation exceeds 180°, the rotation angle calculation unit 243 reverses the rotation direction to point in the calculated direction. In addition, the rotation angle calculation unit 243 calculates a pan angle φ according to the axis deviation of the PTZ camera 20. tra′ When calculating, the angle range that can be taken is -175°≦φ as in the first embodiment. tra′ ≦175° is also taken into consideration.

[0069] 16 shows a case where the x-axis direction of the ceiling microphone 10 and the x-axis direction of the PTZ camera 20, i.e., the direction of the lens of the PTZ camera 20, are the same (when there is no axis misalignment). This shows a case where, for example, the ceiling microphone 10 and the PTZ camera 20 are placed at the front of a room, and the PTZ camera 20 captures images of the room from the front.

[0070] In the upper left diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -175°≦φ tra If the condition of ≦175° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ calculated in step S307 as tra and tilt angle θ tra is used as is.

[0071] In the upper left diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -180°≦φ tra <-175° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the angle range that the PTZ camera 20 can take. tra′ =-175°.

[0072] In the upper left diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, 175°<φ traIf the condition of ≦180° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the angle range that the PTZ camera 20 can take. tra′ = 175°.

[0073] Next, the upper right diagram of FIG. 16 shows a case where the x-axis direction of the ceiling microphone 10 and the x-axis direction of the PTZ camera 20 are opposite directions. In other words, the upper right diagram of FIG. 16 shows a case where the PTZ camera 20 is installed in an orientation rotated +180° with respect to the x-axis direction of the ceiling microphone 10. This shows a case where, for example, the ceiling microphone 10 is placed at the front of a room and the PTZ camera 20 is placed at the rear of the room, and the room is photographed from the rear. In such a case, the rotation angle calculation unit 243 calculates the pan angle φ when there is no axis misalignment. tra The angle corrected to rotate the camera by -180° is called the pan angle φ according to the axis misalignment. tra′ Here, the pan angle φ according to the axis deviation is calculated as follows: tra′ The angular range that can be taken by the pan angle φ is the clockwise angle range from 0° to −175° from the second quadrant to the first quadrant on the xy plane shown in FIG. 3, and the counterclockwise angle range from 0° to 175° from the third quadrant to the fourth quadrant. In other words, the ranges in which panning is not possible are 175° to 180° and −180° to −175°. The rotation angle calculation unit 243 calculates the pan angle φ in this way. tra′ Considering that the possible angle range of is limited, the pan angle φ according to the axis misalignment is calculated as follows: tra′ Calculate.

[0074] In the upper right diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, 5°≦φ tra If the condition of ≦180° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra Calculate -180°. At this time, the calculation result is -175°≦φ tra′ ≦0° (the calculation result is negative), so the pan direction is clockwise.

[0075] In the upper right diagram of FIG. 16, the pan angle φ calculated in step S307tra However, 0°≦φ tra If the condition of <5° is met, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When calculating -180°, the calculation result is -180°≦φ tra′ <-175°. The rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the range of pan angles that the PTZ camera 20 can take. tra′ =-175°. In this case, the calculation result is negative, so the pan direction is clockwise.

[0076] In the upper right diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -5°<φ tra If the condition of <0° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When calculating -180°, the result is -185°<φ tra′ However, since the PTZ camera 20 cannot pan beyond -180°, the rotation direction is reversed by adding 360°. Then, the calculation result is 175°<φ tra′ <180°. The rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the range of pan angles that the PTZ camera 20 can take. tra′ = 175°. In this case, the calculation result is positive, so the pan direction is counterclockwise.

[0077] In the upper right diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -180°≦φ tra If the condition of ≦−5° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When calculating -180°, the result is -360°<φ tra′ However, since the PTZ camera 20 cannot perform panning beyond -180°, the rotation direction is reversed by adding 360°. Then, the calculation result becomes 0°<φ tra′<175°. That is, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra -180°+360°=φ tra Calculate +180°. At this time, the calculation result is positive, so the pan direction is counterclockwise.

[0078] Next, the lower left diagram of FIG. 16 shows a case where the x-axis direction of the PTZ camera 20 is perpendicular to the x-axis of the ceiling microphone 10 so as to approach it. In other words, the lower left diagram of FIG. 16 shows a case where the PTZ camera 20 is installed in a direction rotated +90° with respect to the x-axis direction of the ceiling microphone 10. This shows a case where, for example, the ceiling microphone 10 is placed on the side of a room, and the PTZ camera 20 is placed at the rear of the room, and the room is photographed from behind. In such a case, the rotation angle calculation unit 243 calculates the pan angle φ when there is no axis misalignment. tra The angle corrected to rotate the camera by -90° is the pan angle φ according to the axis misalignment. tra′ Here, the pan angle φ according to the axis deviation is calculated as follows: tra′ The angular range that can be taken by the pan angle φ is the clockwise angular range of 0° to −175° in the first and fourth quadrants on the xy plane shown in Fig. 3, and the counterclockwise angular range of 0° to 175° in the second and third quadrants. In other words, the ranges in which panning is not possible are −180° to −175° and 175° to 180°. The rotation angle calculation unit 243 calculates the pan angle φ in this way. tra′ Considering that the possible angle range of is limited, the pan angle φ according to the axis misalignment is calculated as follows: tra′ Calculate.

[0079] In the lower left diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -85°≦φ tra If the condition of ≦180° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra Calculate -90°. At this time, the calculation result is -175°≦φ tra′ ≦90°. If the result is negative, the pan direction is clockwise, if positive, the pan direction is counterclockwise.

[0080] In the lower left diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -90°≦φ tra If the condition of <-85° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When calculating -90°, the calculation result is -180°≦φ tra′ <-175°, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the pan angle range that the PTZ camera 20 can take. tra′ =-175°. In this case, the calculation result is negative, so the pan direction is clockwise.

[0081] In the lower left diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -95°≦φ tra If the condition <-90° is met, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When calculating -90°, the calculation result is -185°≦φ tra′ <-180°. However, since the PTZ camera 20 cannot pan beyond -180°, the rotation direction is reversed by adding 360°. Then, the calculation result becomes 175°≦φ tra′ <180°. The rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the range of pan angles that the PTZ camera 20 can take. tra′ = 175°. In this case, the calculation result is positive, so the pan direction is counterclockwise.

[0082] In the lower left diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -180°<φ tra If the condition of ≦−95° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When calculating -90°, the calculation result is -270°≦φ tra′However, since the PTZ camera 20 cannot pan beyond -180°, the rotation direction is reversed by adding 360°. Then, the respective calculation results become 90°<φ tra′ <175°. At this time, since the calculation result is positive, the pan direction is counterclockwise. That is, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra -90°+360°=φ tra Calculates +270°.

[0083] Next, the lower right diagram of FIG. 16 shows a case where the x-axis direction of the PTZ camera 20 is perpendicular to the x-axis of the ceiling microphone 10 so as to move away from it. In other words, the lower right diagram of FIG. 16 shows a case where the PTZ camera 20 is rotated -90° with respect to the x-axis direction of the ceiling microphone 10. This shows a case where, for example, the ceiling microphone 10 is placed on the side of a room, and the PTZ camera 20 is placed in the front of the room, and the room is photographed from the front. In such a case, the rotation angle calculation unit 243 calculates the pan angle φ when there is no axis misalignment. tra The angle corrected to rotate +90° from the original position is the pan angle φ according to the axis misalignment. tra′ Here, the pan angle φ according to the axis deviation is calculated as follows: tra′ The angular range that can be taken by the pan angle φ is the clockwise angular range of 0° to −175° in the third and second quadrants on the xy plane shown in FIG. 3, and the counterclockwise angular range of 0° to 175° in the fourth and first quadrants. In other words, the ranges in which panning is not possible are −180° to −175° and 175° to 180°. In this way, the pan angle φ tra′ Considering that the possible angle range of is limited, the pan angle φ according to the axis misalignment is calculated as follows: tra′ Calculate.

[0084] In the lower right diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, 95°≦φ tra If the condition of ≦180° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ traWhen +90° is calculated, the result is 185°≦φ tra′ However, since the PTZ camera 20 cannot pan more than +180°, the rotation direction is reversed by subtracting 360°. Then, the calculation result is -175°<φ tra′ <-90°. In this case, since the calculation result is negative, the pan direction is clockwise. That is, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra +90°-360°=φ tra Calculates -270°.

[0085] In the lower right diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, 90°≦φ tra If the condition of <95° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When calculating +90°, the calculation result is +180°≦φ tra′ However, since the PTZ camera 20 cannot pan more than +180°, the rotation direction is reversed by subtracting 360°. Then, the calculation result is -180°<φ tra′ <-175°. The rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the range of pan angles that the PTZ camera 20 can take. tra′ =-175°. In this case, the calculation result is negative, so the pan direction is clockwise.

[0086] In the lower right diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, 85°≦φ tra If the condition of <90° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra When +90° is calculated, the result is 175°≦φ tra′ <180°. The rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation based on the range of pan angles that the PTZ camera 20 can take. tra′ = 175°. In this case, the calculation result is positive, so the pan direction is counterclockwise.

[0087] In the lower right diagram of FIG. 16, the pan angle φ calculated in step S307 tra However, -180°≦φ tra If the condition of <85° is satisfied, the rotation angle calculation unit 243 calculates the pan angle φ according to the axis deviation. tra′ =φ tra Calculate +90°. The calculation result at this time is -90°≦φ tra′ < 175°. If the result is negative, the pan direction is clockwise, if the result is positive, the pan direction is counterclockwise.

[0088] To summarize the calculation method of the pan angle according to the axis misalignment described above, when the PTZ camera 20 is installed in a direction rotated by Δφ with respect to the x-axis direction of the ceiling microphone 10, the pan angle φ according to the axis misalignment is calculated as follows: tra′ is the pan angle φ calculated in step S307. tra For φ tra′ =φ tra -Δφ. The calculated result φ tra′ When the rotation direction of Δφ is positive, 360° is added to the calculation result, and when the rotation direction of Δφ is negative, 360° is subtracted from the calculation result. tra′ If the calculated angle is outside the range of pan angles that the PTZ camera 20 can take, the camera is panned to the maximum angle in the calculated rotation direction.

[0089] Then, the process proceeds to step S309, and the direction control unit 244 adjusts the pan angle φ calculated in step S307. tra Or the pan angle φ calculated in step S308 tra′ In step S308, the direction control unit 244 controls the pan driving unit 25 based on the pan angle φ tra′ is calculated, the pan angle φ tra′ The direction control unit 244 controls the pan driving unit 25 based on the tilt angle θ calculated in step S307. tra Or the tilt angle θ calculated in step S308 tra′In step S308, the direction control unit 244 controls the tilt driving unit 26 based on the tilt angle θ tra′ is calculated, the tilt angle θ tra′ Based on this, the pan driving unit 25 is controlled. Then, the process proceeds to step S309, where the PTZ camera 20 performs the same process as in step S208 in FIG.

[0090] (Operation and effect of the second embodiment) As described above, the second calculation unit (rotation angle calculation unit) 243 of the information processing device (control unit) 24 according to the second embodiment calculates the coordinates (x tra ,y tra ,z tra ) and the position of the image capturing device 20 relative to the sound detection device (ceiling microphone) 10, the horizontal rotation angle and vertical rotation angle of the image capturing device 20 are calculated.

[0091] Based on the arrangement of the sound detection device and the imaging device, the imaging direction of the imaging device can be calculated taking into account the horizontal and vertical misalignment between the sound detection device and the imaging device, so even if the arrangement of the sound detection device and the imaging device changes, the imaging direction of the imaging device can be accurately controlled to the direction of the sound source.

[0092] Although the embodiments of the present invention have been described above, the descriptions and drawings that form part of this disclosure should not be understood to limit the present invention. Various alternative embodiments, examples, and operating techniques will become apparent to those skilled in the art from this disclosure.

[0093] For example, the multiple functions of the control unit 24 of the PTZ camera 20 in the above embodiment may be realized within the ceiling microphone 10, or may be realized by a microcomputer or the like separate from the ceiling microphone 10 and the PTZ camera 20.

[0094] The sound detection device 10 detects the distance d from the ceiling microphone 10 to the PTZ camera 20. camOr the distance d from the ceiling microphone 10 to the person 30 man Instead of , the vertical distance h from the ceiling microphone 10 to the PTZ camera 20 cam , the vertical distance h from the ceiling microphone 10 to the person 30 man may be calculated.

[0095] The sound detection device 10 detects the distance d from the ceiling microphone 10 to the PTZ camera 20. cam Or the distance d from the ceiling microphone 10 to the person 30 man If it is not possible to calculate the vertical distance h from the ceiling microphone 10 to the PTZ camera 20, any fixed value such as 1.4 (m) or 3 (m) may be used. cam Or the vertical distance h from the ceiling microphone 10 to the person 30 man If it is not possible to calculate the distance d, any fixed value such as 1 (m) or 2 (m) may be used. cam or distance d man , distance h cam or distance h man If fixed values are used for the angle, an error may occur when calculating the position of the person 30 based on the position of the PTZ camera 20. In order to suppress this error, it is desirable to identify the face of the person 30 by analyzing the video captured by the imaging unit 27, or to fine-tune the pan angle and tilt angle by using an automatic tracking function that uses the skeleton of the person 30. [Explanation of symbols]

[0096] 10 Ceiling microphone (voice detection device) 20 PTZ camera (imaging device) 24 Control unit (information processing device) 242 Relative coordinate calculation unit (first calculation unit) 243 Rotation angle calculation unit (second calculation unit) 30 people

Claims

1. a first calculation unit that calculates coordinates of a person based on the position of an imaging device, based on a direction of an imaging device that exists in the same space as the voice detection device and that is detected by the voice detection device and that is based on the position of the voice detection device; and based on a direction of the person that exists in the same space as the voice detection device and that is detected by the voice detection device and that is based on the position of the voice detection device; a second calculation unit that calculates a horizontal rotation angle and a vertical rotation angle of the imaging device based on coordinates of the person relative to a position of the imaging device; An information processing device comprising:

2. the imaging device includes a pan drive unit and a tilt drive unit, a direction control unit that controls the pan drive unit of the imaging device based on the horizontal rotation angle of the imaging device calculated by the second calculation unit, and controls the tilt drive unit based on the vertical rotation angle of the imaging device. The information processing device according to claim 1 .

3. The first calculation unit calculating distances to the imaging device and the person based on the position of the voice detection device, based on the amplitude of the voice signal detected by the voice detection device or a sensor signal of a distance sensor mounted on the voice detection device; Calculating coordinates of the person based on the position of the imaging device, based on the distances to the imaging device and the person based on the position of the voice detection device, the direction of the imaging device based on the position of the voice detection device, and the direction of the person based on the position of the voice detection device.

3. The information processing device according to claim 1.

4. The second calculation unit Calculating a horizontal rotation angle and a vertical rotation angle of the imaging device based on the coordinates of the person calculated by the first calculation unit relative to the position of the imaging device and the position of the imaging device relative to the voice detection device. The information processing device according to claim 2 .

5. An information processing method by an information processing device, calculating coordinates of the person based on the position of the imaging device, based on a direction of the imaging device that exists in the same space as the voice detection device and that is detected by the voice detection device and that is based on the position of the voice detection device, and based on a direction of the person that exists in the same space as the voice detection device and that is detected by the voice detection device and that is based on the position of the voice detection device; Calculating the horizontal and vertical rotation angles of the imaging device based on the calculated coordinates of the person relative to the position of the imaging device. Information processing methods.

Images