Information processing device and control method thereof

Abstract

To enable optimal setting of a main subject during shooting.SOLUTION: A camera includes detection means that detects information about a person, first detection means that detects that a first gesture has been made indicating at least one of a direction, a position, and an area of an object, acquisition means that acquires information about the object in response to the detection of the first gesture, and determination means that determines a main subject in response to a result of acquisition of the information about the object by the acquisition means or a result of detection of the information about the person by the detection means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device and a control method thereof, and more particularly to a technique for assisting in the selection of a main subject. [Background technology]

[0002] In recent years, an increasing number of users are distributing videos in which they film themselves introducing products. In product introduction videos, it is preferable to control the switching of the main subject at an intended timing so that the target product can be emphasized.

[0003] However, in cases where an individual is streaming a video introducing a certain product, the person appearing on screen introducing the product, the person filming it, and the person streaming the video in real time are often the same person. For this reason, it has been difficult to change the settings of the imaging device during video streaming.

[0004] In Patent Document 1, when a subject's movement such as a hand gesture coincides with a pre-stored predetermined movement, the imaging device is made to execute a predetermined process corresponding to the detected predetermined movement, thereby realizing remote control of the imaging device. More specifically, in Patent Document 1, by waving one's arm toward a personal computer or mobile terminal equipped with a digital camera, the shutter can be activated, the zoom can be switched up / down, an e-mail can be sent, or a computer mouse can be operated. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2001-111881 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, no consideration has been given to directly specifying the main subject within the photographing angle of view.

[0007] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to make it possible to optimally set a main subject during shooting. [Means for solving the problem]

[0008] The technical features of the present invention include a detection step of detecting information about a person, a first detection step of detecting that a first gesture has been made indicating at least one of a direction, position, and area of ​​an object, an acquisition step of acquiring information about the object in response to the detection of the first gesture, and a determination step of determining a main subject in response to the acquisition result of the object information by the acquisition step or the detection result of the person information by the detection step. Effect of the Invention

[0009] According to the present invention, the main subject can be optimally set during shooting. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating a hardware configuration of an information processing device according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a functional configuration of an electric circuit 20 according to a first embodiment of the present invention. [Diagram 3] 4 is a flowchart showing a process executed by an electric circuit 20 in the first embodiment of the present invention. [Figure 4] FIG. 4 is a block diagram showing a functional configuration of an electric circuit 20 according to a second embodiment of the present invention. [Diagram 5] 6 is a flowchart showing a process executed by an electric circuit 20 in a second embodiment of the present invention. [Figure 6] FIG. 11 is a block diagram showing a functional configuration of an electric circuit 20 according to a third embodiment of the present invention. [Figure 7] 10 is a flowchart showing a process executed by an electric circuit 20 in a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0012] (First embodiment) Configuration of information processing device 1 is a diagram showing the hardware configuration of an interchangeable lens camera (hereinafter simply referred to as a camera) as an example of an information processing device to which the present invention can be applied. The camera is composed of a lens unit 100 and a camera body 200. When the lens unit 100 is attached to the camera body 200, a lens control unit that controls the operation of the lens unit 100 and a camera control unit that controls the operation of the entire camera can communicate with each other via a communication unit. In addition, the camera body 200 can be connected to a server device 300 on a network by wireless communication or wired communication. The server device 300 is, for example, a video distribution server having a video distribution function.

[0013] The lens unit 100 constitutes a photographing optical system. The lens unit 100 includes an aperture 11, an image stabilization lens group 12, a focus lens / zoom lens group 13, and the like, and can guide an optical image of a subject to the camera body 200.

[0014] The camera body 200 includes an image sensor 21 that photoelectrically converts an optical image formed by the lens unit 100 to generate an image signal. In order to perform focus detection by the image sensor 21 using the image plane phase difference method, each pixel of the image sensor 21 holds a plurality of photodiodes (photoelectric conversion units) for one microlens. In this embodiment, each pixel is configured with two photodiodes arranged in two columns and one row. In each pixel having such a configuration, a light beam that has passed through different pupil regions of the photographing optical system of the lens unit 100 is separated by a microlens and imaged on two photodiodes. Then, a signal (A+B signal) obtained by adding the signals from the two photodiodes is used as an image signal for recording, and two signals (A signal, B signal) read out from each photodiode are used as a focus detection signal. Using the focus detection signal obtained in this manner, focus detection by the image plane phase difference method (image plane phase difference AF) can be performed.

[0015] The camera body 200 includes a mechanical shutter 22 that adjusts the exposure time for exposing the image sensor 21. The camera body 200 controls the aperture 11 and lens groups 12, 13 of the lens unit 100 based on the settings of a plurality of setting items, and also controls the drive timing of the image sensor 21 and the shutter speed of the mechanical shutter 22 to capture an image with appropriate exposure. The camera body 200 includes a rear display unit 23 that can display an image captured by the image sensor 21 and various settings of the device at the time of shooting. The rear display unit 23 is composed of a display device such as a liquid crystal panel or an organic EL, and is provided on the rear part of the camera body 200 on the opposite side to the lens unit 100.

[0016] Note that if the image sensor 21 has an electronic shutter function that can adjust the exposure time by controlling the signal accumulation time and the signal readout time, there is no need for the mechanical shutter 22. Also, if the image sensor 21 has both the mechanical shutter 22 and the electronic shutter function, when adjusting the exposure time with the electronic shutter, the mechanical shutter 22 is set to a fully open state.

[0017] The camera body 200 includes an electric circuit 20. The electric circuit 20 includes an arithmetic processing circuit 20a, a memory circuit 20b, an image processing circuit 20c, an image compression circuit 20d, a drive control circuit 20g, and the like.

[0018] The arithmetic processing circuit 20a includes a processor such as a CPU or MPU that performs various arithmetic processing for controlling the operation of the lens unit 100 and the camera body 200. The arithmetic processing circuit 20a controls each part of the lens unit 100 and the camera body 200 by executing a program stored in the storage unit 29. The program here includes a program that performs the control processing of this embodiment. As an example, the arithmetic processing circuit 20a generates a signal for phase difference AF (a pair of image signals) from a focus detection signal formed by a light beam passing through different pupil regions of the photographing optical system, and detects a defocus amount based on the deviation amount of the pair of image signals. Then, based on the calculated defocus amount DEF, the distance z from the image sensor 21 to the subject is calculated. The distance z can be calculated by the following formulas (1) to (2). dist=1 / (1 / (dist_d+DEF)-1 / f)...(1) z = length - dist (2)

[0019] dist is the distance from the focal position to the subject, dist_d is the distance from the image-side principal point of the lens unit 100 to the image sensor 21 which is the sensor surface, f is the focal length, and length is the distance from the image sensor 21 to the focal position. The focal position corresponds to the in-focus position.

[0020] The distance (length) from the imaging element 21 to the focal position can be measured, for example, by a laser distance measuring means (not shown). By having a data table indicating the relationship between the lens position and the focal position during shooting, it is possible to estimate the distance to the focal position corresponding to the lens position during shooting. The data table indicating the relationship between the lens position and the focal position can reduce the effort required to calculate the distance from the imaging element 21 to the focal position.

[0021] Also, an example of calculating the distance z from the signal output from the imaging element 21 has been described, but the present invention is not limited to this. For example, the distance z can be calculated by a stereo camera. Also, the calculation of these distances z can be performed by an external device of the camera body 200, and the camera body 200 can acquire the distance z calculated by the external device. The distance information to the subject may be acquired by a device such as LiDAR, rather than being calculated from an image. Note that the distance information between the camera body 200 and the subject has been described using the distance from the imaging element 21 as an example, but the present invention is not limited to this, and may be the distance from any position, such as the tip of the lens unit 100.

[0022] The memory circuit 20b is used as a working memory for expanding programs read from the storage unit 29, a buffer memory for temporarily storing image signals captured by the image sensor 21 and focus detection signals, and an image display memory for the rear display unit 23.

[0023] The image processing circuit 20c performs various types of image processing on the image signal generated by the imaging element 21. The image data output from the image processing circuit 20c is output to the rear display unit 23, or is compressed into a predetermined data format by the image compression circuit 20d and output to the storage unit 29 for recording.

[0024] The image compression circuit 20d compresses and encodes the image data output from the image processing circuit 20c into a predetermined data format to generate an image file.

[0025] The drive control circuit 20g controls the drive circuits and actuators (not shown) based on the results of the arithmetic processing by the arithmetic processing circuit 20a, and controls the aperture 11 of the lens unit 100, the lens groups 12 and 13, and the mechanical shutter 22 of the camera body 200.

[0026] The camera body 200 includes an operation input unit 28 such as a switch, a button, or a touch panel that accepts user operations. In this embodiment, the operation input unit 28 includes a shutter switch that instructs preparation for shooting or the start of shooting. By lightly pressing the shutter switch to the first step, a so-called "half press," operations such as autofocus processing, auto exposure processing, and auto white balance processing are started. Furthermore, by pressing the shutter switch deeply from the half press to the second step, a so-called "full press," operations of the shooting processing are started. The shooting processing here refers to a series of processes from activating the mechanical shutter 22 or the electronic shutter function of the image sensor 21 to reading a signal from the image sensor 21 and writing image data to the storage unit 29. As the operation input unit 28, a switch that allows the user to turn on or off a video distribution function by the server device 300 described later may be provided.

[0027] The camera body 200 includes a communication unit 25. The communication unit 25 includes an interface circuit for communicatively connecting the camera body 200 to an external device via a network such as the Internet. The camera body 200 can transmit and receive data to and from an external device connected to a wired or wireless network via the communication unit 25. For example, the camera body 200 can control the communication unit 25 to output image data processed by an image processing circuit to a server device 300 on the network.

[0028] The camera body 200 includes an audio input unit 27. The audio input unit 27 includes a microphone or the like, converts input audio into an electric signal, and outputs it to the electric circuit 20 as audio data. The audio data output to the electric circuit 20 is added to image data and output to the storage unit 29 for recording. In this embodiment, for example, the audio input unit 27 inputs audio uttered by the user and outputs the audio data to the electric circuit 20, thereby enabling audio acquisition. The audio input unit 27 may be built into the camera body 200, or may be connected to an external terminal (not shown).

[0029] The camera body 200 includes a storage unit 29 such as a memory card or a hard disk. The storage unit 29 stores programs executed by the arithmetic processing circuit 20a. The storage unit 29 also records image files compressed into a predetermined format by the image compression circuit 20d, or reads out image files that have already been recorded. The storage unit 29 may be detachable from the camera body 200, or may be built into the camera body 200.

[0030] Next, a description will be given of the configuration and functions of the moving image distribution server 300. The moving image distribution server 300 includes a control unit 30, a communication unit 31, and a streaming processing unit 32.

[0031] The control unit 30 includes a processor such as a CPU or MPU that performs various arithmetic processing to control the operation of the video distribution server 300. The control unit 30 controls each unit of the video distribution server 300 by executing a predetermined program. The program here includes a program that performs the video distribution processing of this embodiment. The communication unit 31 is connected to the communication unit 25 of the camera body 200 via a network, and is capable of sending and receiving data to and from the camera body 200 and an external device. The communication unit 31 outputs image data transmitted from the communication unit 25 of the camera body 200 to the streaming processing unit 32. The streaming processing unit 32 creates a video for distribution based on the image data transmitted from the communication unit 25 and transmits it to the communication unit 31. The image data that has been subjected to the streaming processing is transmitted to a viewer's device, not shown in FIG. 1, via the communication unit 31.

[0032] Functional configuration of the electric circuit 20 in the first embodiment 2 is a block diagram conceptually showing the functional configuration of the electric circuit 20 in the first embodiment. The electric circuit 20 has an image acquisition unit 101, an object detection unit 102, a first focus detection unit 103, a human detection unit 104, a second focus detection unit 105, a priority determination unit 106, a parameter determination unit 107, an automatic focus adjustment unit 108, and a first gesture detection unit 109.

[0033] Each of these processing units is stored as a program in the memory circuit 20b, and is realized by the arithmetic processing circuit 20a executing the program.

[0034] The image acquisition unit 101 sequentially acquires images from the image sensor 21 in real time.

[0035] The object detection unit 102 detects an object from the image acquired by the image acquisition unit 101. The object detected here is a product to be introduced, etc. The object detection unit 102 holds in advance characteristics of the object, such as shape, color, and information, and detects the object from the image based on this information. If there are multiple objects in the image, multiple objects may be detected.

[0036] The first focus detection unit 103 generates a signal for phase difference AF (a pair of image signals) from the focus detection signal in an area corresponding to the object based on the object information acquired by the object detection unit 102, and detects a defocus amount based on the amount of deviation of the pair of image signals. Then, the first focus detection unit 103 calculates a distance z from the image sensor 21 to the object based on the calculated defocus amount DEF.

[0037] The human detection unit 104 detects a human from the image acquired by the image acquisition unit 101. The human detection unit 104 stores shape information of a person's face and body (human body, face, eyes, head, torso, etc.), color information, etc. in advance, and detects a human from the image based on this stored information.

[0038] The second focus detection unit 105 generates a signal for phase difference AF (a pair of image signals) from the focus detection signal in an area corresponding to a person based on the person information detection result by the person detection unit 104, and detects a defocus amount based on the amount of deviation of the pair of image signals. Then, based on the calculated defocus amount DEF, it calculates a distance z from the image sensor 21 to the person.

[0039] The priority order determination unit 106 determines which of the people or objects is to be focused on by focus control. Which object is to be given priority will be described later.

[0040] The parameter determination unit 107 determines focus parameters for focusing the imaging unit on the target using focus detection information calculated by the first focus detection unit 103 and the second focus detection unit 105. The focus parameters are aperture values ​​for adjusting the focus position and the depth of field, etc. The focus position can be expressed by, for example, the distance between the lens and the imaging element, the focal length, etc., depending on the configuration for focusing in the imaging unit 101.

[0041] An automatic focus adjustment unit 108 sets the focus parameters determined by the parameter determination unit 107 in the drive control circuit 20g, and adjusts the focus.

[0042] First gesture detection unit 109 detects a specific gesture from an image acquired by image acquisition unit 101. The gesture detected here is a gesture of pointing a finger in a specific direction, and it is also determined whether an object detected by object detection unit 102 exists in the direction of the finger. Note that the gesture detected may be a gesture indicating the position of a target or a gesture indicating the area of ​​an object, in addition to a gesture of pointing a finger in a specific direction (first detection).

[0043] Focus Parameter Setting Process in the First Embodiment The focus parameter setting process in the first embodiment will be described below with reference to FIG.

[0044] 3 is a flowchart showing an example of the operation of the electric circuit 20 in the first embodiment. In the following description, each step is executed by an arithmetic processing circuit 20a which is a part of the electric circuit.

[0045] When the product introduction mode is started by an input operation of the operation unit input section 28, the image acquisition section 101 of the electric circuit 20 sequentially acquires image signals by the imaging element 21 in step S301.

[0046] In step S302, the object detection unit 102, the human detection unit 104, and the first gesture detection unit 109 start detecting objects, human bodies, and gestures from the acquired image signal. At this time, the parameter determination unit 107 increases the aperture value and deepens the depth of field so that various subjects can be easily detected from the image signal, and the focus is set to a wide range. Step S302 may be executed every time an image is acquired, or may be executed at a first cycle.

[0047] In step S303, the first gesture detection unit 109 determines whether a gesture is detected in the image. If a gesture is detected, the process proceeds to step S304, and if not, the process proceeds to step S306.

[0048] In step S304, it is determined whether an object is detected in the image in the direction of the gesture detected by first gesture detection unit 109. If an object is detected, the process proceeds to step S305, and if not, the process proceeds to step S306.

[0049] In step S305, the first focus detection unit 103 calculates the distance between the image sensor 21 and the object detected in step S304.

[0050] When there are multiple objects, the priority order determination unit 106 determines the object for which the distance to the image sensor 21 is to be calculated, depending on the content of the gesture detected in step S304. For example, when the detected gesture indicates that the nearest object is prioritized, the nearest object in the direction of the finger pointed by the gesture is determined to be prioritized over other objects.

[0051] In step S306, it is determined whether a person has been detected from the image by the human detection unit 104. If a person has been detected, the process proceeds to step S307, and if not, the process proceeds to step S308.

[0052] In step S307, the second focus detection section 103 calculates the distance between the person detected in step S306 and the image sensor 21, based on the image information relating to the person.

[0053] In step S308, the arithmetic processing circuit 20a determines a predetermined subject as the main subject. Then, based on image information related to the main subject, the distance between the main subject and the image sensor 21 is calculated. Here, the predetermined subject is, for example, a subject located near the center of the image. Alternatively, if an object was detected in step S302, it may be determined as the main subject.

[0054] In step S309, the parameter determination unit 107 determines a focus parameter for focusing using the distance calculated in any one of the series of processes. Then, the automatic focus adjustment unit 108 sets the determined focus parameter in the drive control circuit 20g. The series of processes from step S301 to step S309 may be executed repeatedly.

[0055] By processing in this way, the focus can be set on a person if no gesture is detected, and on a product if a gesture is detected, allowing the broadcaster to freely switch the subject on which they focus at the timing they intend.

[0056] In this embodiment, the first gesture detection unit 109 detects a specific gesture from an image, but a gesture may be detected using a millimeter wave sensor instead of an image. In addition, when the device is configured as a compound eye camera, various detection processes may be performed using another camera unit. In addition, in this embodiment, after the main subject is determined, the parameter determination unit determines a focus parameter for focusing for automatic focus adjustment, but the parameter determination unit may determine an exposure time parameter for automatic exposure. In addition, the parameter determination unit may determine a parameter for auto white balance. In addition, the parameter determination unit may determine a parameter for tracking. This makes it possible to realize functions related to automatic focus adjustment, auto white balance, automatic exposure, and tracking.

[0057] Second Embodiment A second embodiment of the present invention will be described below, but the hardware configuration of the device of this embodiment will not be described because it is similar to that of the first embodiment shown in Fig. 1. Fig. 4 is a diagram conceptually showing the functional configuration of an electric circuit 20 in the second embodiment. The configurations of 101 to 109 in Fig. 4 are similar to those of 101 to 109 in Fig. 2, so their description will be omitted.

[0058] The second gesture detection unit 410 and the third gesture detection unit 411 detect a specific gesture from the image acquired by the image acquisition unit 101. While the first gesture detection unit 109 detects a gesture of pointing a finger in a specific direction, the second gesture detection unit 409 detects a gesture of bending a finger (second detection). Furthermore, the third gesture detection unit 410 detects a gesture of pointing with multiple fingers, such as two fingers, or a gesture of waving a hand (third detection).

[0059] The voice detection unit 412 receives voice input by the voice input unit 27 or voice acquired from a video distribution destination (not shown) via the server device 300, and detects a specific voice from among the voices.

[0060] Focus Parameter Setting Process in the Second Embodiment The focus parameter setting process in the second embodiment will be described below with reference to Fig. 5. In step S512, the person detection unit 404 determines whether a person has been detected from the image. At this time, the parameter determination unit 107 increases the aperture value and deepens the depth of field so that various subjects can be easily detected from the image, and focuses on a wide range. If a person has been detected, the process proceeds to step S516, and if not, the process proceeds to step S513.

[0061] In step S513, it is determined whether multiple objects are detected in the image in the direction of the gesture detected by first gesture detection unit 109. If multiple objects are detected, the process proceeds to step S514, and if not, the process proceeds to step S517.

[0062] In step S514, it is determined whether a third gesture has been detected by third gesture detection unit 411 or a second voice has been detected by voice detection unit 412 in the image. The third gesture is a gesture of pointing at an object with multiple fingers or a hand waving motion. The second voice is a voice indicating that the object to be focused on is to be switched. If the third gesture or the second voice is detected, the process proceeds to step S515, and if not, the process proceeds to step S517.

[0063] In step S515, the priority order determination unit 106 determines an object for which the distance to the image sensor 21 is to be calculated, according to the third gesture and the content of the second voice. The distance to the second closest object in the direction of the pointing finger is calculated. By repeating the process of step S515, the priority order determination unit 106 can change the object prioritized by the priority order determination unit 106 to the third closest object, the fourth closest object, and so on.

[0064] In step S516, it is determined whether a second gesture has been detected by second gesture detection unit 410 or a first sound has been detected by sound detection unit 412 in the image. The second gesture is a gesture of pointing a finger toward a person or a gesture of bending a finger. The first sound is a sound indicating that focusing on the person's direction or focusing on an object is to be ended. If the second gesture is detected, the process proceeds to step S518, and if not, the process proceeds to step S517.

[0065] In step S517, the priority order determination unit 106 determines an object for which the distance to the image sensor 21 is to be calculated according to the content of the first gesture. The object closest to the object in the direction of the finger pointed in the gesture is determined as a priority over other objects. In step S518, the second focus detection unit 105 calculates the distance between the person detected in step S512 and the image sensor 21 based on image information about the person. In step S519, the parameter determination unit 107 determines a focus parameter for focusing using the distance calculated in any of the series of processes. When the first gesture detection unit 109 detects multiple gestures and they conflict with each other, one of the following three modes is selected in advance by an input operation of the operation unit input unit 28, and a priority gesture is determined. In mode 1, the gesture detected first is prioritized. In mode 2, a gesture by a person closer to the device is prioritized. In mode 3, a gesture by a person in the direction in which the voice detection unit 412 detects voice is prioritized.

[0066] By processing in this way, the broadcaster can return the focus to the broadcaster at the intended timing after introducing the product by gesture or voice. Also, when there are multiple products, the broadcaster can focus on the product that he / she intended. And when multiple gestures compete, it is possible to decide which gesture to give priority to.

[0067] (Third embodiment) A third embodiment of the present invention will be described below, but the hardware configuration of the device of this embodiment will not be described because it is similar to that of the first embodiment shown in Fig. 1. Fig. 6 is a diagram conceptually showing the functional configuration of an electric circuit 20 in the third embodiment. Note that the configurations of 101 to 108 in Fig. 6 are similar to those of 101 to 108 in Fig. 2, so their description will be omitted.

[0068] The subject estimation unit 610 analyzes the subject in the direction of the gesture detected by the fourth gesture detection unit 609 from the image acquired by the image acquisition unit 101, and estimates the subject to be focused on. For example, if the subject is the sky, the subject to be focused on is estimated to be something flying in the sky, such as a bird or an airplane. If the subject is railroad tracks, the subject to be focused on is estimated to be a train.

[0069] The focus parameter setting process in the third embodiment will be described below with reference to Fig. 7. Steps S701 to S709 of the imaging control process in the third embodiment shown in Fig. 7 correspond to steps S301 to S309 in Fig. 3, so their description will be omitted and only the parts that differ from Fig. 3 will be described.

[0070] In step S704, it is determined whether an object has been detected in the direction of the gesture, and if not, the process proceeds to step S710. In step S710, the object estimation unit 610 estimates the object to be focused on. In step S709, the focus is once set on some object, and then the process proceeds to step S711.

[0071] In step S711, it is determined whether the subject estimated in step S710 is present in the image. If the estimated subject is present in the image, the process proceeds to step S712, and if not, the process ends. In step S712, the first focus detection unit 603 calculates the distance to the image sensor 21 based on image information related to the estimated subject detected in step S712. In step S713, the parameter determination unit 607 determines focus parameters for focusing using the distance calculated in step S712. Then, the automatic focus adjustment unit 108 sets the determined focus parameters in the drive control circuit 20g.

[0072] By processing in this manner, even if the subject that the distributor wants to focus on arrives later, an image can be captured with the appropriate subject in focus.

[0073] In the above-mentioned embodiment, the present invention is applied to an interchangeable lens camera, but the present invention is not limited to this example and can be applied to any device that can control information processing. In other words, the present invention can be applied to mobile phone terminals, PCs, tablet PCs, game consoles, etc.

[0074] (Other embodiments) The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and a computer (or a CPU, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the computer-readable storage medium that stores the program constitute the present invention. [Explanation of symbols]

[0075] 100 Lens section 11 Aperture 12 Image stabilization lenses 13 Focus / zoom lenses 200 Camera body 20 Electrical Circuits 21 Image sensor 22 Mechanical Shutter 23 Rear display 25 Camera body communication section 27 Audio input section 28 Operation input section 29 Memory section 300 Server device 30 Control section 31 Server device side communication unit 32 Streaming Processing Unit

Claims

1. A detection means for detecting information about a person, A first detection means for detecting that a first gesture has been made indicating at least one of the direction, position, or area of ​​an object, An acquisition means for acquiring information about the object in response to the detection of the first gesture, An information processing apparatus comprising: a determination means for determining the main subject according to the acquisition result of information about the object by the acquisition means or the detection result of information about a person by the detection means.

2. The information processing apparatus according to claim 1, characterized in that when the first gesture is detected by the first detection means, the determination means determines the main subject based on the acquisition result of the object information by the acquisition means.

3. The information processing apparatus according to claim 1, characterized in that, if the first gesture is not detected by the first detection means, the determination means determines the main subject based on the information of the person detected by the detection means.

4. The information processing device according to claim 2, characterized in that if the first gesture is no longer detected after determining the main subject based on the information of the object, the main subject is determined based on the information of the person detected by the detection means.

5. The system further includes a second detection means for detecting that a second gesture has been made, in which the person has bent their finger in the direction of the person detected by the detection means, or in which the person has pointed their finger in the direction of the person. The information processing apparatus according to claim 1 or 2, characterized in that when the second gesture is detected, the main subject is determined based on the information of the person detected by the detection means.

6. The system further comprises a third detection means for detecting that a third gesture has been made. The information processing apparatus according to claim 1, characterized in that the determination means changes the main subject when the third gesture is detected.

7. The information processing device according to claim 6, characterized in that the third gesture is a gesture of pointing with multiple fingers in the direction of the person detected by the detection means, or waving a hand.

8. The system further includes a voice detection means for detecting specific sounds, The information processing apparatus according to claim 1, characterized in that when the voice detection means detects the specific voice, the determination means determines the main subject based on the information of the person detected by the detection means.

9. Furthermore, it includes a means for acquiring audio specific to the video streaming source, The information processing apparatus according to claim 1, characterized in that the specific sound indicates that the main subject has been determined to be a person, or that the determination of the main subject to be an object has been completed.

10. Further comprising estimation means for estimating the main subject, The information processing apparatus according to claim 1, wherein the determination means determines the main subject based on the information of the person detected by the detection means if there is no subject in the direction of the first gesture, if the estimated subject exists in the image, the estimated subject is used as the main subject, and if the subject does not exist in the image, the main subject is determined based on the information of the person detected by the detection means.

11. The information processing apparatus according to claim 1, characterized in that the detection means detects at least one of the human body, face, pupil, head, or torso.

12. The information processing apparatus according to claim 1, further comprising control means for controlling the execution of a specific function based on the main subject determined by the determination means.

13. The information processing device according to claim 12, characterized in that the aforementioned specific function is at least one of the functions relating to autofocus, auto white balance, auto exposure, and tracking.

14. The information processing apparatus according to claim 1, further comprising an imaging means, wherein the first detection means detects the first gesture from an image obtained by the imaging means.

15. The information processing apparatus according to claim 5, further comprising an imaging means, wherein the second detection means detects the second gesture from an image obtained by the imaging means.

16. The information processing apparatus according to claim 6, further comprising an imaging means, wherein the third detection means detects the third gesture from an image obtained by the imaging means.

17. A detection process for detecting information about a person, A first detection step of detecting that a first gesture has been made indicating at least one of the direction, position, or area of ​​an object, An acquisition step of acquiring information about the object in response to the detection of the first gesture, A control method for an information processing device, characterized by comprising: a determination step of determining the main subject according to the acquisition result of information about the object by the acquisition step or the detection result of information about a person by the detection step.

18. A program for causing a computer to execute each step of the control method described in claim 17.

19. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method described in claim 17.

Images