Speaker position detection method, speaker position detection device, and program

Abstract

To provide a speaker position detection method capable of significantly calculating a speaker position.SOLUTION: A speaker position detection method includes the steps of: acquiring voices of a speaker by a microphone; calculating direction information of the speaker based on the acquired voices of the speaker; acquiring a face image of the speaker from an image, which is acquired by a camera, on the basis of the direction information; and detecting position information including height direction of the speaker based on the acquired face image of the speaker.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] An embodiment of the present invention relates to a speaker position detection method, a speaker position detection device, and a program. [Background technology]

[0002] Patent document 1 discloses a method for determining a speaker's position, which detects a person's position by processing an input signal from one of the sensors, such as a television camera, an ultrasonic sensor, or an infrared sensor, detects a sound source position by processing an input signal from a microphone array, and determines the speaker's position by processing the two types of information together. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-41577 Summary of the Invention [Problem to be solved by the invention]

[0004] The determination method of Patent Document 1 does not take into account height information of the speaker, and therefore cannot uniquely determine the speaker's position on the camera image.

[0005] Therefore, an object of one embodiment of the present invention is to provide a speaker position detection method capable of uniquely determining a speaker position. [Means for solving the problem]

[0006] The speaker position detection method includes acquiring the speaker's voice with a microphone, determining directional information of the speaker based on the acquired voice of the speaker, acquiring a facial image of the speaker from an image acquired by a camera based on the directional information, and detecting position information including height information of the speaker based on the acquired facial image of the speaker. Effect of the Invention

[0007] According to an embodiment of the present invention, the speaker position can be uniquely determined. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a speaker position detection device 1. FIG. [Diagram 2] FIG. 2 is an elevation view of a room in which the talker position detection device 1 is installed. [Diagram 3] FIG. 1 is a plan view of a room in which a talker position detection device 1 is installed. [Figure 4] 4 is a flowchart showing the operation of a control unit 12. [Diagram 5] FIG. 4 is a diagram illustrating an example of a user interface displayed on a display device. [Figure 6] FIG. 13 is a diagram showing an example of a user interface according to the first modified example. [Figure 7] FIG. 13 is a diagram illustrating an example of a user interface when a non-focus region is displayed. [Figure 8] FIG. 11 is a diagram showing an example of a user interface according to Modification 2. [Figure 9] FIG. 13 is a diagram showing an example of a user interface according to Modification 3. [Figure 10] FIG. 13 is a diagram showing an example of a user interface according to Modification 4. [Figure 11] FIG. 13 is an elevation view of a room in which a talker position detection device 1 in a fifth modified example is installed. [Figure 12] FIG. 13 is an elevational plan view of a room in which a talker position detection device 1 in a fifth modified example is installed. [Figure 13] FIG. 23 is a diagram showing an example of a user interface according to the 11th modification. [Figure 14] FIG. 23 is a diagram showing an example of a user interface according to the 11th modification. [Figure 15] FIG. 4 is a diagram illustrating an example of a user interface displayed on a display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Fig. 1 is a block diagram showing the configuration of the talker position detection device 1. Fig. 2 is a schematic elevation view of a room in which the talker position detection device 1 is installed, and Fig. 3 is a schematic plan view.

[0010] The speaker position detection device 1 includes a camera 11, a control unit 12, a flash memory 14, a RAM 15, a user interface (I / F) 16, a speaker 17, a plurality of microphones 18A to 18D, and a communication I / F 19.

[0011] The talker position detection device 1 is installed on the ceiling of a room. The talker position detection device 1 has a housing in the shape of a thin rectangular parallelepiped. In this embodiment, the talker position detection device 1 is installed on the ceiling. The housing has a camera 11, a speaker 17, and microphones 18A to 18D arranged therein.

[0012] A desk is placed directly below the housing of the speaker position detection device 1. In the examples of Fig. 2 and Fig. 3, a plurality of users (users u1, u2) are present around the desk.

[0013] In the present invention, it is not essential that camera 11, speaker 17, and microphones 18A to 18D are installed on the ceiling. For example, in the present invention, microphones 18A to 18D may be installed on the ceiling, and camera 11 and speaker 17 may be set on a desk.

[0014] The camera 11 captures an image of the user. The microphones 18A to 18D capture the voice of the user. The speaker 17 outputs the voice to the user.

[0015] In this embodiment, the number of microphones is four, forming a microphone array, although the number of microphones is not limited to four.

[0016] The control unit 12 functions as a control unit that comprehensively controls the operation of the speaker position detection device 1 by reading an operating program from the flash memory 14 into the RAM 15. For example, the flash memory 14 stores a program 141. The control unit 12 executes the speaker position detection method of the present invention by using the program 141. It is not necessary to store the program in the flash memory 14 of the device itself. The control unit 12 may download the program each time from a server or the like and read it into the RAM 15.

[0017] The control unit 12 also functions as a signal processing unit that processes video signals and audio signals. The control unit 12 performs pan, tilt, and zoom processing (hereinafter referred to as PTZ processing) to enlarge the face image of a speaker, for example. The control unit 12 may also perform processing to mask the images of non-participants, for example.

[0018] The control unit 12 also performs beamforming directivity processing. Beamforming is processing for forming a sound collection beam with high sensitivity in the direction of the speaker by aligning the phase in the direction of the speaker using delay and sum processing, for example.

[0019] The communication I / F 19 transmits the video signal and audio signal processed by the control unit 12 to another device. The other device is, for example, an information processing terminal such as a personal computer used by a user. The information processing terminal (near-end information processing terminal) is connected to a remote information processing terminal (far-end information processing terminal) via a network such as the Internet. The near-end information processing terminal transmits the video signal and audio signal received from the speaker position detection device 1 to the far-end information processing terminal. The near-end information processing terminal also receives the video signal and audio signal from the far-end information processing terminal. The near-end information processing terminal outputs the received video signal to a display device (not shown). The near-end information processing terminal also transmits the received audio signal to the speaker position detection device 1. The communication I / F 19 of the speaker position detection device 1 outputs the received audio signal to the speaker 17. The speaker 17 outputs sound related to the received audio signal. In this way, the speaker position detection device 1 functions as a component of a remote conference system for holding a conference with a remote location.

[0020] Fig. 4 is a flowchart showing the operation of the control unit 12. First, the control unit 12 acquires the voice of the speaker from the microphones 18A to 18D (S11). The control unit 12 obtains directional information of the voice of the speaker (S12). In the examples of Figs. 2 and 3, the control unit 12 obtains directional information of the voice of the user u1.

[0021] The direction information includes an elevation angle φ with a vertical downward direction being 0°, and a horizontal angle θ with a reference direction being 0° in a planar view. The control unit 12 analyzes the sound signals acquired from the microphones 18A to 18D to estimate the direction from which the sound comes. The method of analyzing the sound signals may be any method, such as a cross-correlation method, a delay-and-sum method, or a multiple signal classification (MUSIC) method.

[0022] In the cross-correlation method, the control unit 12 calculates the cross-correlation of sound signals from, for example, multiple microphones. The control unit 12 obtains the cross-correlation peak of sound signals from, for example, two microphones. Furthermore, the control unit 12 obtains the cross-correlation peak of sound signals from another two microphones. The control unit 12 estimates the sound arrival direction based on the multiple cross-correlation peaks calculated in this manner. That is, the control unit 12 selects two or more sets of multiple microphones and obtains multiple cross-correlation peaks. The estimated sound arrival direction is represented by, for example, a space vector. The control unit 12 compares the estimated sound arrival direction with a vertical downward direction to obtain an elevation angle φ. Furthermore, the control unit 12 compares the estimated sound arrival direction with a reference direction to obtain a horizontal angle θ.

[0023] Next, the control unit 12 acquires a face image of the speaker from the image acquired by the camera 11 based on the obtained direction information (S13). The face image is acquired by processing to recognize the face of the speaker from the image acquired by the camera 11 using a predetermined algorithm such as a neural network. However, the control unit 12 performs the speaker face recognition processing only on a portion of the image acquired by the camera 11 that corresponds to the elevation angle φ and horizontal angle θ obtained in the processing of S12.

[0024] Then, the control unit 12 detects position information including the speaker's height information based on the acquired face image of the speaker (S14). The control unit 12 detects the position information including the speaker's height information, for example, by using a trained model that trains the relationship between the position and size of the speaker's face image on the image acquired by the camera 11 and the speaker's height information from the floor. Alternatively, the control unit 12 may detect the position information including the speaker's height information by referring to a table or function that specifies the relationship between the position and size of the speaker's face image on the image acquired by the camera 11 and the speaker's height information from the floor.

[0025] Furthermore, the control unit 12 may obtain distance information from the speaker using a trained model that trains the relationship between the position and size of the face image of the speaker on the image acquired by the camera 11 and distance information from the speaker. Alternatively, the control unit 12 may obtain distance information from the speaker by referring to a table or function that specifies the relationship between the position and size of the face image of the speaker on the image acquired by the camera 11 and distance information from the speaker. When the control unit 12 obtains distance information from the speaker, the control unit 12 may convert the distance information into height information of the speaker using height information from the floor surface to the ceiling surface.

[0026] In this manner, the control unit 12 detects position information including the speaker's directional information (the elevation angle φ and the horizontal angle θ) and height information. In face recognition processing by conventional image processing, it is necessary to recognize the speaker's face from the entire image acquired by the camera 11, and to distinguish between the faces of non-speakers (non-speakers) and speakers. However, the control unit 12 of this embodiment first acquires the speaker's directional information based on a sound signal, and therefore performs face recognition processing of the speaker by focusing on a portion of the image acquired by the camera 11 that corresponds to the speaker's directional information. In addition, the control unit 12 of this embodiment does not need to distinguish between the faces of non-speakers (non-speakers) and speakers. As a result, the control unit 12 of this embodiment significantly reduces the load related to face recognition processing, can acquire the speaker's face image at high speed and with high accuracy, and can uniquely determine the speaker's position on the image acquired by the camera 11.

[0027] The speaker's position information including the height information is displayed, for example, on a display of the near-end information processing terminal. FIG. 5 is a diagram showing an example of a user interface displayed on a display of the near-end information processing terminal. The user interface of FIG. 5 includes a schematic plan view of the conference room viewed from the top and a schematic side view of the conference room viewed from the side. The user interface of FIG. 5 is used by a user to set the position of the focus of the sound collection beam. The focus position of the sound collection beam is an example of a focus area of ​​a microphone. The user interface of FIG. 5 shows the height and direction of the position that is the focus of the sound collection beam. In the example of FIG. 5, the user sets the direction of the focus of the sound collection beam to 135° and the height to 1.6 m.

[0028] The control unit 12 displays the detected speaker position information on the display. In the example of Fig. 5, the direction information of the detected speaker position information is a horizontal angle of 130°, and the height information is 1.0 m. The control unit 12 displays the speaker direction information (horizontal angle 130°) and height information (1.0 m) on the user interface, superimposed on each other.

[0029] This allows the user to easily confirm that the horizontal angle of the set sound collection beam is off by +5° and the height is off by +0.6m. In other words, the user can obtain a new customer experience in which they can easily determine whether the sound collection beam settings are correct or not.

[0030] In addition, by changing the horizontal angle of the set sound collection beam by -5° and the height by -0.6m, the user can focus the sound collection beam on the speaker's position. In other words, the user can get a new customer experience of being able to easily modify the sound collection beam settings.

[0031] (Variation 1) The focus area of ​​the microphone is not limited to the focal position of the sound collection beam. The focus area of ​​the microphone may be the operating range of the sound collection beam. Fig. 6 is a diagram showing an example of a user interface according to the first modification.

[0032] In the example of Fig. 6, the control unit 12 displays the operating range of the sound collection beam as the focus area of ​​the microphone. The operating range of the sound collection beam is set by the user via the user I / F 16, for example. The sound collection beam is configured so that the focal position can be set only within the set operating range. In this case, only the voice of the speaker in the focus area is picked up, and the voice of the speaker in the non-focus area other than the focus area is not picked up.

[0033] Therefore, the user can easily determine the area from which the sound is picked up, providing a new customer experience.

[0034] The control unit 12 may display the non-operating range of the sound collection beam as the non-focus region of the microphone. Fig. 7 is a diagram showing an example of a user interface when the non-focus region is displayed. In the example of Fig. 7, the control unit 12 displays the non-operating range of the sound collection beam as the non-focus region of the microphone.

[0035] In this case, the user can easily determine areas where sound is not being picked up, providing a new customer experience.

[0036] Furthermore, the control unit 12 may receive the focus area of ​​the camera or microphone, and acquire an image of the speaker based on the focus area corresponding to the directional information of the speaker among the received focus areas. Specifically, as shown in Fig. 15, when the control unit 12 receives settings of the first to fourth focus areas as the focus areas of the camera or microphone, the control unit 12 performs face recognition processing of the speaker in the focus area corresponding to the directional information of the speaker (the fourth focus area in the example of Fig. 15). This allows the control unit 12 to significantly reduce the load related to the face recognition processing, acquire a face image of the speaker at high speed and with high accuracy, and uniquely determine the position of the speaker on the image acquired by the camera 11.

[0037] (Variation 2) The control unit 12 of the second modification obtains a focus area of ​​the microphone based on the position information of the detected speaker, and displays the obtained focus area on the display. Fig. 8 is a diagram showing an example of a user interface according to the second modification.

[0038] The control unit 12 obtains an operating range of the sound collecting beam as a focus area within a predetermined range around the detected speaker position (for example, a square-shaped range centered on the speaker position in a plan view), and displays the obtained focus area. In this case, the operating range of the sound collecting beam is set to a predetermined range around the detected speaker position.

[0039] According to the second modification, the operating range of the sound collecting beam is set around the speaker. Therefore, the user can obtain a new customer experience in which the operating range of the sound collecting beam does not need to be manually set.

[0040] (Variation 3) The control unit 12 of the third modification obtains a focus area of ​​the microphone based on the position information of the detected speaker, and performs focus control of the microphone based on the obtained focus area. Fig. 9 is a diagram showing an example of a user interface according to the third modification.

[0041] The control unit 12 sets the focal position of the sound collecting beam as a focus area at the position of the detected speaker, and also displays the focal position of the sound collecting beam set as the focus area.

[0042] According to the third modification, the focal position of the sound collection beam is automatically set to the position of the speaker. Therefore, only the voice of the speaker is collected with a high signal-to-noise ratio. In addition, the user can obtain a new customer experience in that the focal position of the sound collection beam does not need to be set manually. In addition, the user can obtain a new customer experience in that the focal position of the sound collection beam can be directed to the user's position simply by speaking, for example.

[0043] (Variation 4) The focus control of the microphone includes a control for masking the voices of the person other than the speaker. The control unit 12 of the fourth modification obtains a non-focus area of ​​the microphone based on the position information of the detected speaker, and performs a control for masking the voices of the person other than the speaker based on the obtained non-focus area. Fig. 10 is a diagram showing an example of a user interface according to the fourth modification.

[0044] The control unit 12 obtains an operating range of the sound collecting beam as a focus area within a predetermined range around the position of the detected speaker. The control unit 12 also displays a non-operating range of the sound collecting beam as a non-focus area outside the focus area, as shown by the hatched area in FIG.

[0045] In this case, only the voice of the speaker in the focus area is picked up, and the voice of the speaker in the non-focus area outside the focus area is not picked up and is masked. Therefore, only the voice of the speaker is picked up with an even higher signal-to-noise ratio. In addition, the user can obtain a new customer experience in that the area where the voice is not picked up can be easily determined. In addition, the user can obtain a new customer experience in that the user does not need to manually set the non-focus area where the voice is not picked up.

[0046] (Variation 5) Fig. 11 is an elevation view of a room in which a talker position detection device 1 in the modification 5 is installed. Fig. 12 is an elevation view of a room in which a talker position detection device 1 in the modification 5 is installed.

[0047] The microphones 18A to 18D of the talker position detection device 1 according to the fifth modification are installed on the ceiling of a room, and the camera 11 is installed on a desk. The other configuration is the same as the configuration shown in FIGS.

[0048] Control unit 12 acquires the speaker's voice from microphones 18A to 18D, and obtains directional information of the speaker's voice. Then, control unit 12 acquires relative position information between camera 11 and the microphones (microphones 18A to 18D), and acquires a face image of the speaker from the image acquired by camera 11 based on the relative position information and directional information.

[0049] The relative position information is expressed, for example, by a spatial vector. In the examples of Fig. 11 and Fig. 12, the camera 11 is at the same left-right position (x-axis direction in Fig. 12) as the microphone when viewed from above (Fig. 12), and is above the microphone when viewed from above (y-axis direction in Fig. 12). Also, the height of the camera 11 is lower than that of the microphone when viewed from above (Fig. 11). Therefore, for example, the spatial vector of the relative position information of the microphone as viewed from the camera is expressed as (x, y, z) = (0, -a, +b). The control unit 12 obtains a spatial vector (speaker position coordinates) indicating the direction of the speaker from the camera 11 based on the relative position information and the spatial vector of the arrival direction of the speaker's voice based on the collected sound. In the examples of Figures 11 and 12, the spatial vector of the sound arrival direction estimated based on the sound picked up by the microphone is expressed as (x, y, z) = k (ex, ey, ez), where ex, ey, and ez are the unit vectors of the x-axis, y-axis, and z-axis, respectively (k is a coefficient that represents the magnitude according to distance).

[0050] The control unit 12 acquires an image at a position (x, y, z) = (0 + kex, -a + key, +b + kez) by the camera 11 while changing the coefficient k of the unit vector (for example, changing the distance in real space to about 30 cm to 3 m), and identifies the speaker (for example, a face) from the acquired image to specify the coefficient k of the speaker position. For example, when a speaker's face is detected in an image with coefficient k = k3, the spatial vector indicating the speaker's position from the camera 11 is expressed as (x, y, z) = (0 + k3 · ex, -a + k3 · ey, +b + k3 · ez).

[0051] The control unit 12 performs face recognition processing of the speaker only on the portion of the image acquired by the camera 11 that corresponds to the spatial vector to the speaker's position.

[0052] In this way, the speaker position detection device 1 of this embodiment can significantly reduce the load associated with face recognition processing, acquire a speaker's facial image quickly and with high accuracy, and uniquely determine the speaker's position on the image acquired by camera 11, even if the camera 11 and the microphones (microphones 18A to 18D) are located in different positions.

[0053] (Variation 6) The control unit 12 according to the sixth modification obtains the elevation angle, horizontal angle, and magnification amount of the camera 11 based on the direction information, relative position information, and height information in the fifth modification. The control unit 12 performs PTZ processing as an example of focus control based on the obtained elevation angle, horizontal angle, and magnification amount. The PTZ processing may be performed by physically controlling the shooting direction of the camera 11 or by controlling an optical zoom lens, or may be performed by applying signal processing to the image acquired from the camera 11. This automatically focuses on the face image of the speaker.

[0054] In addition, when the camera 11 and the microphones 18A to 18D are arranged in the same housing as shown in Figures 2 and 3, when the control unit 12 determines the position information of the speaker, the control unit 12 may perform PTZ processing using directional information (horizontal angle θ and elevation angle φ) determined based on the speaker's voice and distance information determined based on the speaker's image.

[0055] (Variation 7) The control unit 12 according to the seventh modification masks the face images of persons other than the speaker. The masking includes processes such as blurring, filling in, or replacing with another image. Alternatively, the control unit 12 may mask the entire non-focus area of ​​the image of the camera 11. This allows the control unit 12 to not output images of persons other than the participants of an event such as a conference, and can output natural images while maintaining the privacy of the non-participants.

[0056] (Variation 8) The position information of the speaker is not limited to the position information of one speaker, and may include the position information of multiple speakers. The control unit 12 of this embodiment first acquires the direction information of the speaker based on the sound signal, and performs face recognition processing of the speaker by focusing on the part of the image acquired by the camera 11 that corresponds to the direction information of the speaker. In addition, the control unit 12 does not need to distinguish and recognize the faces of the speaker from those of non-speakers (non-speakers). Therefore, the control unit 12 of this embodiment significantly reduces the load related to the face recognition processing. Therefore, even when detecting the position information of multiple speakers, the control unit 12 of the modified example 8 can acquire the face images of the speakers at high speed and with high accuracy, and can uniquely identify the positions of the multiple speakers.

[0057] (Variation 9) The control unit 12 in the ninth variant determines the focus area of ​​the camera or microphone multiple times based on the position information of the detected speaker, and determines an average focus area or average non-focus area based on the focus area or non-focus area determined multiple times.

[0058] In this case, the focus area or non-focus area according to the speaker's position is gradually optimized as time passes from the start of an event such as a conference. In addition, the user can obtain a new customer experience in which the focus area and non-focus area do not need to be manually set.

[0059] (Variation 10) The control unit 12 of the modification 12 detects the movement of the speaker from the image captured by the camera. The movement of the speaker is, for example, the direction of the face or a gesture. The control unit 12 detects the direction of the face of the speaker from the image captured by the camera 11.

[0060] The control unit 12 detects a specific gesture from an image captured by the camera 11, for example, by using a trained model that has been trained to have a correspondence relationship between an image and a movement related to the gesture.

[0061] The control unit 12 controls the focus of the camera or microphone according to the detected speaker's movement. For example, if the speaker turns his / her face to the right, the control unit 12 changes the direction of the sound collection beam to the right as seen by the user. Also, for example, if the speaker makes a gesture to cover his / her eyes, the control unit 12 masks the face image of the speaker.

[0062] The control unit 12 of this embodiment first acquires the speaker's direction information based on the sound signal, and then performs processing to detect the speaker's movement by focusing on the part of the image acquired by the camera 11 that corresponds to the speaker's direction information. Therefore, the control unit 12 of this embodiment significantly reduces the load related to the processing to detect the speaker's movement. Therefore, even when performing complex processing such as detecting the speaker's movement, the control unit 12 of the modified example 10 can detect the speaker's movement at high speed and with high accuracy, and control the focus of the camera or microphone according to the movement. In addition, the user can have a new customer experience of being able to control the direction of the sound collection beam and the image without speaking.

[0063] (Variation 11) The control unit 12 of variant example 11 obtains first position information of a first speaker and second position information of a second speaker, obtains a first focus area of ​​a camera or microphone based on the first position information, obtains a second focus area of ​​a camera or microphone based on the second position information, and sets a third focus area that encompasses the first focus area and the second focus area if certain conditions are satisfied.

[0064] 13 and 14 are diagrams showing an example of a user interface according to Modification 11. As shown in Fig. 13, the control unit 12 sets a first focus area F1 based on first position information of a first speaker S1. Also, the control unit 12 sets a second focus area F2 based on second position information of a second speaker S2.

[0065] The predetermined condition is, for example, that the set focus regions overlap. In the example of Fig. 13, the first focus region F1 and the second focus region F2 overlap in plan view. Therefore, the control unit 12 sets a third focus region F3 that includes the first focus region F1 and the second focus region F2, as shown in Fig. 14.

[0066] As a result, for example, when multiple speakers are conversing, one third focus area F3 that encompasses these multiple speakers is set, so that even if the speaker is frequently changed, it is not necessary to change the focus area each time. Therefore, the control unit 12 can further reduce the load related to the process of detecting the speaker position and the focus control.

[0067] (Variation 12) The control unit 12 of the modification 12 detects the start and end of an event. For example, the control unit 12 detects the start and end of an event when an operation to start or end the event is received via the user I / F 16, or when an operation to turn the power on or off is received via the user I / F 16.

[0068] When the control unit 12 detects the end of an event, it resets the position information. When the control unit 12 detects the start of an event, it obtains directional information of the speaker based on the speaker's voice acquired at that time, acquires a face image of the speaker from the image acquired by the camera 11 based on the directional information, and detects position information based on the acquired face image of the speaker. In other words, the control unit 12 resets the position information at the end of an event, and detects new position information at the start of a new event.

[0069] This allows the control unit 12 to determine the speaker's position anew for each event, even if the environment changes for each event, and to optimize the focus area or non-focus area according to the speaker's position as the event, such as a conference, progresses.

[0070] (Variation 13) The control unit 12 of the thirteenth modification detects the environment of the event based on the sound captured by the microphone and the image captured by the camera.

[0071] The event environment includes, for example, the location where the meeting will be held, the number of participants and their locations, outdoors, indoors, private rooms, semi-private rooms, etc. This allows users to obtain a new customer experience in which, for example, the locations of participants do not need to be manually set in advance.

[0072] The description of the present embodiment should be considered as illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, not by the above-described embodiments. Furthermore, the scope of the present invention includes the scope equivalent to the claims.

[0073] For example, the above-mentioned events are not limited to conferences, but also include live performances such as concerts. [Explanation of symbols]

[0074] 1: speaker position detection device, 11: camera, 12: control unit, 14: flash memory, 15: RAM, 16: user I / F, 17: speaker, 18A to 18D: microphones, 19: communication I / F, 141: program

Claims

1. The speaker's voice is picked up by a microphone, Obtaining direction information of the speaker based on the acquired voice of the speaker; acquiring a face image of the speaker based on the direction information and an image acquired by a camera; detecting position information including height information of the speaker based on the acquired face image of the speaker; Speaker location detection method.

2. displaying the detected speaker's position information on a display device; The speaker position detection method according to claim 1 .

3. Accepting a focus area of ​​the camera or the microphone; acquiring an image of the speaker based on a focus area corresponding to the direction information among the received focus areas; The speaker position detection method according to claim 1 or 2.

4. A focus area of ​​the camera or the microphone is obtained based on the detected position information of the speaker. displaying the obtained focus area or a non-focus area other than the focus area on a display device; The speaker position detection method according to claim 1 or 2.

5. A focus area of ​​the camera or the microphone is obtained based on the detected position information of the speaker. performing focus control of the camera or the microphone based on the determined focus area; The speaker position detection method according to claim 1 or 2.

6. The focus control of the camera or the microphone includes control for masking face images or voices of persons other than the speaker. The speaker position detection method according to claim 5.

7. Obtaining relative position information of the camera and the microphone; determining a direction of the speaker in an image of the camera based on the relative position information and the direction information; acquiring a face image of the speaker from the image acquired by the camera based on a direction of the speaker in the image of the camera; The speaker position detection method according to claim 1 or 2.

8. Obtaining relative position information of the camera and the microphone; the directional information includes an elevation angle and a horizontal angle from the microphone; determining an elevation angle, a horizontal angle, and a magnification amount of the camera based on the direction information, the relative position information, and the height information; The speaker position detection method according to claim 1 or 2.

9. The speaker position information includes position information of a plurality of speakers. The speaker position detection method according to claim 1 or 2.

10. Detect the start and end of an event, resetting the location information when the end of the event is detected; When the start of the event is detected, direction information of the speaker is obtained based on the voice of the speaker acquired at that time, a face image of the speaker is obtained from an image acquired by a camera based on the direction information, and position information including height information of the speaker is detected based on the acquired face image of the speaker. The speaker position detection method according to claim 1 or 2.

11. Detecting an environment of the event based on the audio captured by the microphone and the images captured by the camera; The speaker position detection method according to claim 10.

12. determining a focus area of ​​the camera or the microphone multiple times based on the detected position information of the speaker; determining an average focus area or an average non-focus area based on a plurality of the focus areas or non-focus areas other than the focus areas; The speaker position detection method according to claim 1 or 2.

13. Detecting the speaker's movements from the image captured by the camera; performing focus control of a camera or a microphone in response to the detected movement of the speaker; The speaker position detection method according to claim 1 or 2.

14. Obtaining first location information of a first speaker and second location information of a second speaker; determining a first focus area of ​​the camera or the microphone based on the first position information; determining a second focus area of ​​the camera or the microphone based on the second position information; when a predetermined condition is satisfied, a third focus area including the first focus area and the second focus area is set. The speaker position detection method according to claim 1 or 2.

15. With a microphone Camera and A control unit; Equipped with The control unit is Acquiring the speaker's voice with the microphone; Obtaining direction information of the speaker based on the acquired voice of the speaker; acquiring a face image of the speaker based on the direction information and the image acquired by the camera; detecting position information including height information of the speaker based on the acquired face image of the speaker; Speaker location detection device.

16. The speaker's voice is picked up by a microphone, Obtaining direction information of the speaker based on the acquired voice of the speaker; acquiring a face image of the speaker based on the direction information and an image acquired by a camera; detecting position information including height information of the speaker based on the acquired face image of the speaker; A program that causes an information processing device to execute processing.

Images