Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program with which it is possible to easily obtain a captured image with an appropriate angle of view using fewer capturing devices. The information processing device is provided with: an angle of view candidate setting unit which sets a plurality of angle of view candidates from a captured image range that can be captured by a first capturing device with respect to a predetermined capturing space, and a capturing angle of view setting unit which, on the basis of the plurality of angle of view candidates, sets a capturing angle of view for the first capturing device. The present technology is applicable to, for example, a capturing system for television broadcast programs and the like.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that enable an image with an appropriate angle of view to be easily obtained with a small number of imaging devices. [Background technology]

[0002] 2. Description of the Related Art Conventionally, in the field of program production such as television broadcasting, images with various angles of view are captured using many imaging devices so that images with appropriate angles of view can be provided to viewers (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-101165 A Summary of the Invention [Problem to be solved by the invention]

[0004] On the other hand, in recent years, video distribution platforms have become more diverse, and there are an increasing number of cases where video production and video distribution are carried out by a small number of people or by people without specialized knowledge. Accordingly, there is an increasing demand for technology that can easily obtain images with an appropriate angle of view using a small number of shooting devices.

[0005] The present technology has been made in consideration of such circumstances, and makes it possible to easily obtain images with an appropriate angle of view using a small number of imaging devices. [Means for solving the problem]

[0006] According to one aspect of the present technology, there is provided an information processing device, comprising: a view angle candidate setting unit that sets a plurality of view angle candidates from a photographic image range that can be photographed by a first photographing device for a predetermined photographing space; and a photographing view angle setting unit that sets a photographing view angle of the first photographing device based on the plurality of view angle candidates. a view angle control unit that controls pan, tilt, and zoom of the first image capturing device so that the image capturing angle becomes the set view angle, and controls driving speeds of pan, tilt, and zoom of the first image capturing device based on whether a predetermined role is assigned to an image captured by the first image capturing device;Equipped with.

[0007] According to an aspect of the present technology, there is provided an information processing method, in which an information processing device sets a plurality of candidate angles of view from a range of captured images that can be captured by an image capturing device for a predetermined image capturing space. To do and setting a photographing angle of view of the photographing device based on the plurality of candidate angles of view. controlling the pan, tilt, and zoom of the photographing device so that the photographing angle of view becomes the set photographing angle; and controlling the driving speeds of the pan, tilt, and zoom of the photographing device based on whether or not a predetermined role has been assigned to an image photographed by the photographing device.

[0008] A program according to one aspect of the present technology includes a step of setting a plurality of candidate angles of view from a range of captured images that can be captured by a photographing device for a predetermined photographing space. To do and setting a photographing angle of view of the photographing device based on the plurality of candidate angles of view. The image capturing device is caused to execute a process including controlling the pan, tilt, and zoom of the image capturing device so that the image capturing angle becomes the set image capturing angle, and controlling the driving speeds of the pan, tilt, and zoom of the image capturing device based on whether or not a predetermined role has been assigned to an image captured by the image capturing device.

[0009] In one aspect of the present technology, a plurality of angle of view candidates are set from a photographic image range that can be photographed by a photographing device for a predetermined photographing space, and a photographing angle of view of the photographing device is set based on the plurality of angle of view candidates. The pan, tilt, and zoom of the photographing device are controlled so that the photographing angle of view becomes the set angle, and the driving speeds of the pan, tilt, and zoom of the photographing device are controlled based on whether or not a specified role has been assigned to the image photographed by the photographing device. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of a shooting scene. [Diagram 2] 1 is a block diagram showing a first embodiment of an information processing system to which the present technology is applied. [Diagram 3] FIG. 2 is a block diagram showing an example of the configuration of an imaging device. [Figure 4] 1 is a block diagram showing a first embodiment of an information processing unit of an information processing device; [Diagram 5] 4 is a flowchart for explaining a first embodiment of a view angle control process. [Figure 6] FIG. 2 is a diagram showing examples of feature points of a person; [Figure 7] 11A and 11B are diagrams for explaining a method of setting a candidate angle of view and a photographing angle of view. [Figure 8] FIG. 13 is a diagram showing an example of a full-angle image. [Figure 9] FIG. 13 is a diagram showing an example of a cut-out image. [Figure 10]FIG. 11 is a block diagram showing a second embodiment of an information processing system to which the present technology is applied. [Figure 11] FIG. 11 is a block diagram showing a second embodiment of an information processing unit of an information processing device. [Figure 12] 10 is a flowchart illustrating a second embodiment of the view angle control process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, modes for carrying out the present technology (hereinafter, referred to as embodiments) will be described. The description will be given in the following order. 1. Background 2. First embodiment 3. Second embodiment 4. Variations 5.Other

[0012] <<1.Background>> As described above, conventionally, in the field of program production for television broadcasting and the like, images with various angles of view are captured using many imaging devices. Then, images to be used for broadcasting and the like are appropriately switched from among the images with the various angles of view.

[0013] For example, A in Fig. 1 shows an example of a scene from a news program. In this example, three people, Newscaster 1, Commentator 2, and Commentator 3, are lined up in order from the right. In addition, a monitor 4 is placed between Newscaster 1 and Commentator 2.

[0014] In this case, for example, it is assumed that at least images will be captured that show individual bust shots of news anchor 1, commentator 2, and commentator 3, an image of adjacent commentators 2 and 3 taken together (group shot), and an image of the entire scene.

[0015] 1B shows an example of a talk show scene, in which three guests, a host 11, a guest 12, and a guest 13, are lined up from the right.

[0016] In this case, for example, it is expected that at least images will be captured that take bust shots of the presenter 11, guest 12, and guest 13 individually, an image (group shot) of adjacent guests 12 and 13 taken together, and an image of the entire scene.

[0017] 1C shows an example of a scene from a variety show. In this example, the presenter 21 is standing at the far left. To the right of the presenter 21, guests 22 to 24 are sitting in the first row of a stage, and guests 25 and 26 are sitting in the second row of the stage.

[0018] In this case, for example, it is expected that at least images will be taken of the presenter 21 and guests 22 to 26 taken individually in bust shots, an image of adjacent guests 22 to 26 taken together (group shot), and an image of the entire scene.

[0019] If images with different angles of view are captured by different camera devices, the cost and the number of cameramen required will increase, and the control of the camera devices will become complicated.

[0020] In response to this, a method has been used in the past to obtain images with multiple angles of view using a single image capture device. Known methods for obtaining images with multiple angles of view using a single image capture device include, for example, a cropping method and a PTZ control method.

[0021] The cropping method is a method in which a wide range is photographed with the PTZ (pan, tilt, and zoom) of the imaging device fixed, and an image with a required angle of view is cropped from the photographed image. For example, in the example of A in Fig. 1, a photographed image overlooking the entire scene is photographed, and from the obtained photographed image, images such as a bust shot of each person are cropped individually.

[0022] The cropping method allows images with various angles of view to be obtained simultaneously. In addition, because there are no physical constraints, the angle of view can be switched instantly.

[0023] On the other hand, when using the cropping method, there is a risk of image quality (resolution) decreasing when trying to take a close-up shot of a small subject, and there is also a risk of motion blur occurring when the subject is moving quickly.

[0024] The PTZ control method is a method of shooting by driving the PTZ of the imaging device to physically move the angle of view of the imaging device.

[0025] The PTZ control method drives the PTZ, making it possible to increase the variety of angles of view.

[0026] On the other hand, in the PTZ control method, the camera needs to be physically moved to change the angle of view (subject or composition), and it takes time to obtain an image with the required angle of view. Therefore, for example, the subject may move before the angle of view can be adjusted, and an image with the desired angle of view may not be obtained.

[0027] As described above, each method has its advantages and disadvantages, and the advantages and disadvantages are in a trade-off relationship.

[0028] In response to this, the present technology overcomes the shortcomings of both methods and makes it possible to easily obtain images with an appropriate angle of view using a small number of image capturing devices.

[0029] <<2. First embodiment>> First, a first embodiment of the present technology will be described with reference to FIGS.

[0030] <Configuration of Information Processing System 101> FIG. 2 shows an example of the configuration of an information processing system 101 according to a first embodiment of the present technology.

[0031] The information processing system 101 includes a wide-angle camera 111, a microphone 112, a PTZ camera 113, an information processing device 114, and a switcher 115. The wide-angle camera 111, the microphone 112, the PTZ camera 113, the information processing device 114, and the switcher 115 are connected to each other via a wired and / or wireless network, and transmit and receive various types of data.

[0032] The wide-angle camera 111 is, for example, configured with an imaging device capable of wide-angle imaging. The wide-angle camera 111 is installed, for example, in a location where it can broadly view (overlook) the imaging space, and performs wide-angle imaging so as to include a wide area of ​​the imaging space. The wide-angle camera 111 transmits image data (hereinafter referred to as wide-angle image data) corresponding to an image obtained by wide-angle imaging (hereinafter referred to as wide-angle image) to the information processing device 114.

[0033] Here, the shooting space is a three-dimensional space in which the information processing system 101 is shot (is the subject of shooting), for example, a space in which an event to be shot (for example, program production, a sports match, etc.) takes place.

[0034] The PTZ and position of the wide-angle camera 111 are usually fixed.

[0035] The microphone 112 collects sounds within the imaging space and transmits audio data corresponding to the collected sounds to the information processing device 114 .

[0036] The PTZ camera 113 is configured, for example, by an imaging device whose PTZ can be remotely controlled. The angle of view of the PTZ camera 113 is controlled by driving the PTZ under the control of the information processing device 114. The PTZ camera 113 captures an image of an imaging space and transmits image data corresponding to the captured image to the information processing device 114.

[0037] Incidentally, the position of the PTZ camera 113 is usually fixed.

[0038] The information processing device 114 is composed of, for example, a computer. The information processing device 114 calibrates the wide-angle camera 111 and the PTZ camera 113. The information processing device 114 also recognizes the situation of the shooting space based on the wide-angle image data and the audio data. For example, the state of the subject in the shooting space, the content of the scene in the shooting space (hereinafter referred to as the shooting scene), the content of the sound and the position of the sound source in the shooting space, the state of the event taking place in the shooting space, etc. are recognized as the situation of the shooting space.

[0039] The information processing device 114 drives the PTZ of the PTZ camera 113 according to the situation of the shooting space, thereby controlling the angle of view of the PTZ camera 113. Furthermore, the information processing device 114 cuts out an image with a necessary angle of view from an image captured by the PTZ camera 113, and transmits image data (hereinafter referred to as cut-out image data) corresponding to the cut-out image (hereinafter referred to as cut-out image) to the switcher 115. Furthermore, the information processing device 114 transmits wide-angle image data to the switcher 115.

[0040] The switcher 115 selects a necessary image from the wide-angle image and the cut-out image, and transmits image data corresponding to the selected image to a downstream stage. In addition, the switcher 115 cuts out an image with a necessary angle of view from the wide-angle image as necessary, and transmits image data corresponding to the cut-out image to a downstream stage.

[0041] <Example of configuration of information processing device 114> FIG. 3 shows an example of the hardware configuration of the information processing device 114.

[0042] In the information processing device 114, a CPU 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are interconnected via a bus 204. An input / output interface 205 is further connected to the bus 204. An input unit 206, an output unit 207, a storage unit 208, a communication unit 209, and a drive 210 are connected to the input / output interface 205.

[0043] The input unit 206 includes, for example, a keyboard, a mouse, a microphone, and the like.

[0044] The output unit 207 includes, for example, a display, a speaker, and the like.

[0045] The storage unit 208 includes, for example, a non-volatile memory such as a hard disk.

[0046] The communication unit 209 communicates with the wide-angle camera 111, the microphone 112, the PTZ camera 113, and the switcher 115 using a predetermined communication method.

[0047] The drive 210 drives a removable recording medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0048] The program executed by the information processing device 114 (CPU 201) can be provided by being recorded on a removable recording medium 211 such as a package medium. The program recorded on the removable recording medium 211 is installed in the storage unit 208 via the input / output interface 205 by, for example, mounting the removable recording medium 211 on the drive 210.

[0049] The program is also received by the communication unit 209 via a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting, and is installed in the storage unit 208.

[0050] Furthermore, the program can be pre-installed in, for example, the ROM 202 or the storage unit 208.

[0051] In the information processing device 114, for example, the CPU 201 loads a program stored in the ROM 202 or the storage unit 208 into the RAM 203 via the input / output interface 205 and the bus 204, and executes the program, thereby performing a series of processes.

[0052] In the following description, the bus 204 and the input / output interface 205 will be omitted when each unit in the information processing device 114 transmits and receives data via the bus 204 and the input / output interface 205. For example, when the CPU 201 and the communication unit 209 communicate via the bus 204 and the input / output interface 205, the description of the bus 204 and the input / output interface 205 will be omitted, and it will simply be described that the CPU 201 and the communication unit 209 communicate.

[0053] <Configuration example of the information processing unit 251> FIG. 4 shows an example of the configuration of an information processing unit 251 which is a first embodiment of the functions realized by the CPU 201 of the information processing device 114 executing a program.

[0054] The information processing unit 251 includes a calibration unit 261, an imaging control unit 262, and a cropping unit 263.

[0055] The calibration unit 261 calibrates the wide-angle camera 111 and the PTZ camera 113. For example, the calibration unit 261 obtains the positions and orientations of the wide-angle camera 111 and the PTZ camera 113 in a world coordinate system common to the wide-angle camera 111 and the PTZ camera 113. Specifically, the calibration unit 261 obtains a transformation matrix that transforms the world coordinate system into a camera coordinate system specific to the wide-angle camera 111 and the PTZ camera 113. The camera coordinate systems are coordinate systems that use the optical axes of the wide-angle camera 111 and the PTZ camera 113 as references. The calibration unit 261 supplies information related to the transformation matrices of the wide-angle camera 111 and the PTZ camera 113 to the shooting control unit 262.

[0056] The shooting control unit 262 controls the PTZ of the PTZ camera 113, thereby controlling the angle of view of the PTZ camera 113. The shooting control unit 262 includes a recognition unit 271, an angle of view setting unit 272, and an angle of view control unit 273.

[0057] The recognition unit 271 recognizes the situation of the shooting space based on the wide-angle image data from the wide-angle camera 111 and the audio data from the microphone 112. The recognition unit 271 supplies information on the recognized situation of the shooting space to the angle of view setting unit 272.

[0058] The angle of view setting unit 272 sets an angle of view (hereinafter referred to as an imaging angle of view) at which the PTZ camera 113 images the imaging space based on the situation of the imaging space. The angle of view setting unit 272 includes a candidate angle of view setting unit 281 and an imaging angle of view setting unit 282.

[0059] The view angle candidate setting unit 281 sets a plurality of view angle candidates from the range of images that can be captured by the PTZ camera 113 based on the situation of the shooting space. The view angle candidate is, for example, an angle of view predicted to be possibly used in broadcasting, etc. The view angle candidate setting unit 281 supplies information on the set view angle candidates to the shooting view angle setting unit 282.

[0060] The imaging angle of view setting unit 282 sets the imaging angle of view based on the angle of view candidate. The imaging angle of view setting unit 282 supplies information on the imaging angle of view to the angle of view control unit 273. In addition, the imaging angle of view setting unit 282 supplies information on the imaging angle of view and the angle of view candidate included in the imaging angle of view to the cropping unit 263.

[0061] The angle-of-view control unit 273 drives the PTZ of the PTZ camera 113 to control the angle of view of the PTZ camera 113. The angle-of-view control unit 273 also supplies information indicating the state of the angle of view of the PTZ camera 113 to the cropping unit 263.

[0062] The cropping unit 263 crops out an image with a required angle of view from an image captured by the PTZ camera 113, and transmits cropped image data corresponding to the cropped image (cropped image) to the switcher 115 via the communication unit 209.

[0063] <First embodiment of the angle of view control process> Next, a first embodiment of the view angle control process executed by the information processing device 114 will be described with reference to the flowchart of FIG.

[0064] This process is started, for example, when shooting of the shooting space is started, and is ended when shooting of the shooting space is ended.

[0065] Furthermore, it is assumed that prior to this processing, calibration unit 261 calibrates wide-angle camera 111 and PTZ camera 113, and that the transformation matrices of wide-angle camera 111 and PTZ camera 113 are known. In other words, it is assumed that the positions and orientations of wide-angle camera 111 and PTZ camera 113 in the world coordinate system are known.

[0066] In step S1, the recognition unit 271 recognizes the situation of the shooting space.

[0067] Specifically, the recognition unit 271 acquires wide-angle image data from the wide-angle camera 111 and acquires audio data from the microphone 112 via the communication unit 209. The recognition unit 271 recognizes the situation of the shooting space by performing image recognition on the wide-angle image data and audio recognition on the audio data.

[0068] For example, the recognition unit 271 recognizes the state of a subject in a wide-angle image. More specifically, the recognition unit 271 recognizes subjects present in a wide-angle image by using a method such as face detection, face recognition, posture recognition, behavior recognition, and object detection, and also recognizes the type, position, posture, movement, feature points, and the like of each subject.

[0069] FIG. 6 shows examples of feature points to be detected when the subject is a person.

[0070] Feature point Pf1L corresponds to the left eye, and feature point Pf1R corresponds to the right eye. Feature point Pf2L corresponds to the left ear, and feature point Pf2R corresponds to the right ear. Feature point Pf3 corresponds to the nose. Feature point Pf4 corresponds to the neck joint. Feature point Pf5L corresponds to the left shoulder joint, and feature point Pf5R corresponds to the right shoulder joint. Feature point Pf6L corresponds to the left elbow joint, and feature point Pf6R corresponds to the right elbow joint. Feature point Pf7L corresponds to the left wrist joint, and feature point Pf7R corresponds to the right wrist joint. Feature point Pf8L corresponds to the left hip joint, and feature point Pf8R corresponds to the right hip joint. Feature point Pf9L corresponds to the left knee joint, and feature point Pf9R corresponds to the right knee joint. Feature point Pf10L corresponds to the left ankle joint, and feature point Pf10R corresponds to the right ankle joint.

[0071] For example, by detecting and tracking feature points including a person's joints in this manner, the person's skeleton, posture, movement, etc. can be recognized.

[0072] The recognition unit 271 also recognizes the contents of the photographed scene using a technique such as scene recognition. Furthermore, the recognition unit 271 recognizes a main subject (hereinafter referred to as a main subject) and subjects related to the main subject (hereinafter referred to as related subjects) based on the state of the subject and the contents of the photographed scene.

[0073] For example, if the content of the shooting scene is a talk show, the main person in the talk (e.g., the speaker, guest, person at the center of the topic, etc.), the monitor, the flip, etc. may be recognized as the main subject. In addition, for example, a person listening to the main person, the main person's partner, the flip, etc. may be recognized as the related subject.

[0074] For example, if the content of the shooting scene is a live music concert, the main people in the concert (such as the vocalist or the guitarist playing the guitar solo) may be recognized as the main subject. Also, for example, other members, co-stars, instruments, etc. other than the main people may be recognized as related subjects.

[0075] For example, if the content of the shooting scene is a sports broadcast, the main person in the sport (e.g., the player holding the ball, the leading player, etc.) may be recognized as the main subject. Also, for example, players other than the main person, the ball, etc. may be recognized as related subjects.

[0076] Furthermore, the recognition unit 271 recognizes, for example, the content of the sound in the shooting space and the position of the sound source.

[0077] Furthermore, the recognition unit 271 recognizes the state of an event taking place in the shooting space based on, for example, the state of the subject, the content of the shooting scene, the content of the sound in the shooting space, and the position of the sound source.

[0078] The recognition unit 271 supplies information relating to the recognized state of the shooting space to the angle of view setting unit 272 .

[0079] In step S2, the view angle candidate setting unit 281 sets a view angle candidate based on the state of the shooting space. For example, the view angle candidate setting unit 281 sets a view angle candidate based on the main subject, related subjects, and the contents of the shooting scene.

[0080] For example, if the content of the shooting scene is a talk show, the candidate angles of view are set to be an angle of view that includes the part above the chest of the main subject (bust shot), and an angle of view that includes a combination of the main subject and each related subject.

[0081] For example, if the content of the shooting scene is a live music concert, the candidate angles of view are set to include an angle of view that includes the main subject, i.e., the part of the main person above the waist (waist shot), an angle of view that includes the entire body (full figure), and an angle of view that includes a combination of the main subject and each related subject.

[0082] For example, when the content of the photographed scene is a sports match, the angle of view including the entire body of the main subject (full figure), as well as the angle of view including a combination of the main subject and each related subject, are set as the candidate angles of view.

[0083] Fig. 7 shows an example of setting a candidate angle of view. In this example, an example of a wide-angle image of a talk show is shown typically. This wide-angle image shows people 301 to 304. Note that the auxiliary lines superimposed on people 301 to 304 in this figure show the results of face recognition and skeletal recognition.

[0084] A person 301 stands near the right edge of the wide-angle image, and people 302 to 304 are sitting side by side in the range from near the center to the left edge of the wide-angle image.

[0085] Here, for example, it is assumed that person 302 is speaking and person 303 is listening to person 302. In this case, for example, person 302 is recognized as a main subject and person 303 is recognized as a related subject. Then, for example, a candidate angle of view Ac1 including a part above the chest of person 302 and a candidate angle of view Ac2 including a part above the waist of person 302 and person 303 are set.

[0086] The view angle candidate setting unit 281 supplies information on the set view angle candidates to the imaging view angle setting unit 282 .

[0087] In step S3, the imaging angle of view setting unit 282 calculates an image frame that includes all the image angle candidates. For example, the imaging angle of view setting unit 282 calculates a minimum rectangular frame that includes all the image angle candidates as the image frame.

[0088] In step S4, the imaging angle of view setting unit 282 adds a margin to the calculated image frame. For example, the imaging angle of view setting unit 282 expands the image frame by adding a margin in at least one direction around the calculated image frame based on the respective movements of the main subject and related subjects. At this time, the margin in the direction in which the movements of the main subject and related subjects are large becomes large, and the margin in the direction in which the movements of the main subject and related subjects are small becomes small.

[0089] In step S5, the imaging angle of view setting unit 282 determines whether or not the cut-out size of the main angle of view candidate is equal to or larger than a threshold value.

[0090] Specifically, the imaging angle of view setting unit 282 selects the angle of view candidate that is estimated to be the most important from among the angle of view candidates (hereinafter, referred to as the main angle of view candidate) based on the size, position, posture, movement, etc. of the main subject and related subjects, the contents of the imaging scene, and the use of the image, etc. Here, the use of the image is assumed to be, for example, television broadcasting, web distribution, production of video software, etc.

[0091] Next, the imaging angle of view setting unit 282 calculates the resolution (number of pixels) of the main angle of view candidate when the PTZ camera 113 captures an image at the imaging angle of view corresponding to the calculated image frame.

[0092] Next, the imaging angle setting unit 282 compares the calculated resolution with a predetermined threshold. This threshold varies depending on, for example, the use of the image. For example, the threshold is set to 1920 pixels x 1080 pixels (FHD) when the use of the image is television broadcasting, and is set to 1280 pixels x 720 pixels (HD) when the use of the image is web distribution.

[0093] Then, when the resolution of the main angle of view candidate is less than the threshold, the imaging angle of view setting unit 282 determines that the cut-out size of the main angle of view candidate is less than the threshold, and the process proceeds to step S6.

[0094] In step S6, the imaging angle of view setting unit 282 excludes the angle of view candidate closest to the edge of the image frame from among the angle of view candidates other than the main angle of view candidate.

[0095] Thereafter, the process returns to step S3, and steps S3 to S6 are repeatedly executed until it is determined in step S5 that the cutout size of the main angle of view candidate is equal to or larger than the threshold value.

[0096] As a result, the angle of view candidates other than the main angle of view candidate are deleted in order from the edge of the image frame until the cutout size (resolution) of the main angle of view candidate becomes equal to or larger than the threshold value, thereby making the image frame smaller. As a result, the angle of view candidates included in the shooting angle of view are selected.

[0097] On the other hand, in step S5, if the resolution of the main angle of view candidate is equal to or greater than the threshold, the imaging angle of view setting unit 282 determines that the clipping size of the main angle of view candidate is equal to or greater than the threshold, and the process proceeds to step S7.

[0098] In step S7, the imaging angle of view setting unit 282 sets the calculated image frame as the imaging angle of view. As a result, an area with a margin provided around a rectangular area including the selected angle of view candidate is set as the imaging angle of view. The imaging angle of view setting unit 282 supplies information about the set imaging angle of view to the angle of view control unit 273. In addition, the imaging angle of view setting unit 282 supplies information about the set imaging angle of view and the angle of view candidates included in the imaging angle of view to the angle of view control unit 273.

[0099] For example, in the example of FIG. 7, a photographing angle of view As is set that includes the angle of view candidates Ac1 and Ac2 and has a margin around it.

[0100] In step S8, the angle-of-view control unit 273 determines whether or not the image captured by the PTZ camera 113 is being used. For example, when a predetermined role is assigned to the image captured by the PTZ camera 113, for example, when an image cut out from the captured image is output from the switcher 115 as an image for broadcast or preview, the angle-of-view control unit 273 determines that the image captured by the PTZ camera 113 is being used, and the process proceeds to step S9.

[0101] In step S9, the angle-of-view control unit 273 performs low-speed control of the PTZ camera 113 so that the imaging angle of view becomes the set angle of view. Specifically, the angle-of-view control unit 273 drives the PTZ of the PTZ camera 113 at a slow speed to match the angle of view of the PTZ camera 113 to the set angle of view for imaging.

[0102] In this process, the PTZ drive speed of the PTZ camera 113 is set based on, for example, the shutter speed of the PTZ camera 113, the allowable amount of motion blur, etc. In addition, the allowable amount of motion blur is set based on, for example, the content of the shooting scene (e.g., sports broadcast, studio shooting, etc.) and the use of the image (e.g., television broadcast, web distribution, video software production, etc.). For example, if the content of the shooting scene is a scene in which the subject moves a lot, such as sports, the allowable amount of motion blur is set to a large value. On the other hand, if the content of the shooting scene is a scene in which the subject moves a little, such as studio shooting, the allowable amount of motion blur is set to a small value.

[0103] As a result, the angle of view of the PTZ camera 113 is set to an angle of view that includes all the selected candidate angles of view (hereinafter, referred to as the full angle of view).

[0104] For example, as described above, in the example of FIG. 7, by setting the imaging angle of view As, a full-angle image including the person 302 and the person 303 is captured as shown in FIG.

[0105] Furthermore, even if the angle of view of the PTZ camera 113 is changed, motion blurring that occurs in images captured during use is suppressed.

[0106] Then, the process proceeds to step S11.

[0107] On the other hand, in step S8, if a specified role has not been assigned to the image cut out from the captured image captured by the PTZ camera 113, for example, if the image cut out from the captured image is not output from the switcher 115 as an image for broadcast or preview, etc., the angle of view control unit 273 determines that the image captured by the PTZ camera 113 is not being used, and the process proceeds to step S10.

[0108] In step S10, the angle-of-view control unit 273 performs high-speed control of the PTZ camera 113 so that the angle of view of the PTZ camera 113 becomes the set angle of view of the camera. Specifically, the angle-of-view control unit 273 drives the PTZ of the PTZ camera 113 at high speed so that the angle of view of the PTZ camera 113 becomes the set angle of view of the camera in the shortest time. For example, the PTZ is driven at the maximum speed allowed by the mechanical constraints of the PTZ camera 113.

[0109] This allows the angle of view of the PTZ camera 113 to be quickly set to the full angle of view.

[0110] Then, the process proceeds to step S11.

[0111] In step S11, the cropping unit 263 crops out a captured image. Specifically, the cropping unit 263 acquires captured image data captured at a full angle of view (hereinafter referred to as full angle of view image data) from the PTZ camera 113. Then, the cropping unit 263 crops out images of angles of view corresponding to each of the candidate angles of view from the full angle of view image corresponding to the full angle of view image data.

[0112] For example, as shown in FIG. 9, an image with an angle of view that corresponds to the candidate angle of view Ac1 in FIG. 7 and includes a bust shot of person 302 is cut out from the full angle of view image in FIG.

[0113] At this time, when a plurality of view angle candidates are set, the cutout unit 263 cuts out images having view angles corresponding to all the view angle candidates.

[0114] Furthermore, the angle of view of the cut-out image and the corresponding candidate angle of view do not necessarily have to match completely. For example, the angle of view of the cut-out image may be adjusted so that the subject is appropriately included.

[0115] The cut-out unit 263 transmits cut-out image data corresponding to the cut-out image to the switcher 115 via the communication unit 209 .

[0116] Thereafter, the process returns to step S1, and the processes from step S1 onwards are executed.

[0117] As described above, by effectively combining the PTZ method and the cropping method, an image with an appropriate angle of view and image quality can be obtained with a small number of imaging devices. That is, since the angle of view candidates that may be used are predicted in advance and imaging is performed with an imaging angle of view that includes all of the predicted angle of view candidates, an image with a required angle of view can be obtained at an appropriate timing. In addition, since the resolution of the main angle of view candidates is set to a predetermined threshold value or higher, an image with good image quality can be obtained.

[0118] <<3. Second embodiment>> Next, a second embodiment of the present technology will be described with reference to FIGS.

[0119] <Configuration of Information Processing System 401> Fig. 10 shows a configuration example of an information processing system 401 according to a second embodiment of the present technology. In the figure, the same reference numerals are used to designate parts corresponding to those in the information processing system 101 in Fig. 2, and the description thereof will be omitted as appropriate.

[0120] An information processing system 401 in FIG. 10 differs from the information processing system 101 in FIG. 2 in that the wide-angle camera 111 is omitted.

[0121] In the information processing system 401, the PTZ camera 113 captures both a wide-angle image and a full-angle image. The information processing device 114 recognizes the situation of the shooting space based on the wide-angle image or the full-angle image captured by the PTZ camera 113 and audio data, and controls the shooting angle of view of the PTZ camera 113.

[0122] Here, a wide-angle image is an image captured by setting the PTZ camera 113 to a wide viewing angle. The wide viewing angle is an angle of view that allows a wide view of the shooting space, for example, an angle of view that maximizes the field of view of the PTZ camera 113. For example, the wide viewing angle is an angle of view that allows a wide view of an event taking place in the shooting space, and is an angle of view that includes as many subjects (people and objects) related to the event as possible.

[0123] The angle of view at which the field of view is maximum means the angle of view at which the field of view is maximum within a predetermined setting range of PTZ camera 113, and is not necessarily the angle of view at which the field of view of PTZ camera 113 is mechanically maximum. For example, when the setting range of the field of view of PTZ camera 113 is set to a range smaller than the maximum mechanical field of view of PTZ camera 113, the angle of view at which the field of view is maximum within that setting range is set as the wide field of view angle.

[0124] <Configuration example of the information processing unit 451> Fig. 11 shows a configuration example of an information processing unit 451 which is a second embodiment of the function realized by the CPU 201 of the information processing device 114 executing a program. In the figure, the same reference numerals are used to denote parts corresponding to the information processing unit 251 in Fig. 3, and the description thereof will be omitted as appropriate.

[0125] The information processing unit 451 is similar to the information processing unit 251 in that it is provided with a calibration unit 261 and a cropping unit 263, and differs in that it is provided with an imaging control unit 461 instead of the imaging control unit 262. The imaging control unit 461 is similar to the imaging control unit 262 in that it is provided with an angle of view control unit 273, and differs in that it is provided with a recognition unit 471 and an angle of view setting unit 472 instead of the recognition unit 271 and the angle of view setting unit 272. The angle of view setting unit 472 is similar to the angle of view setting unit 272 in that it is provided with an angle of view candidate setting unit 281, and differs in that it is provided with a imaging angle of view setting unit 481 instead of the imaging angle of view setting unit 282.

[0126] Since the information processing system 401 is not provided with the wide-angle camera 111 , the calibration unit 261 only calibrates the PTZ camera 113 .

[0127] The recognition unit 471 recognizes the situation of the shooting space based on the wide-angle image or the full-angle image captured by the PTZ camera 113 and the sound collected by the microphone 112. The recognition unit 471 supplies information on the recognized situation of the shooting space to the angle-of-view setting unit 472.

[0128] The imaging angle of view setting unit 481 sets the imaging angle of view based on the angle of view candidate. Also, the imaging angle of view setting unit 481 sets the imaging angle of view to a wide field of view angle at a predetermined timing. The imaging angle of view setting unit 481 supplies information on the imaging angle of view to the angle of view control unit 273. Also, the imaging angle of view setting unit 481 supplies information on the imaging angle of view and the angle of view candidate included in the imaging angle of view to the cropping unit 263.

[0129] <Second embodiment of the angle of view control process> Next, a second embodiment of the view angle control process executed by the information processing device 114 will be described with reference to the flowchart of FIG.

[0130] This process is started, for example, when shooting of the shooting space is started, and is ended when shooting of the shooting space is ended.

[0131] In step S101, the information processing unit 451 performs high-speed control of the PTZ camera 113 so that the angle of view becomes wide. Specifically, the imaging angle of view setting unit 481 sets the imaging angle of view to a wide angle of view, and supplies information on the set imaging angle of view to the angle of view control unit 273. The angle of view control unit 273 drives the PTZ of the PTZ camera 113 at high speed so that the angle of view of the PTZ camera 113 becomes the wide angle of view at the shortest possible time.

[0132] As a result, the angle of view of the PTZ camera 113 is set to a wide field of view angle, and the PTZ camera 113 captures wide-angle images of the shooting space.

[0133] In step S102, the recognition unit 471 recognizes the situation of the shooting space. Specifically, the recognition unit 471 acquires wide-angle image data or full-angle image data from the PTZ camera 113 via the communication unit 209, and acquires audio data from the microphone 112. Then, the recognition unit 471 recognizes the situation of the shooting space based on the wide-angle image data or full-angle image data and the audio data by the same process as in step S1 of Fig. 5. The recognition unit 471 supplies information related to the recognized situation of the shooting space to the angle-of-view setting unit 472.

[0134] In step S103, the imaging angle of view setting unit 481 determines whether or not the main subject is captured. If the imaging angle of view setting unit 481 determines based on the recognition result of the situation of the imaging space that the main subject is captured in the full-angle image captured by the PTZ camera 113, that is, if the imaging angle of view setting unit 481 determines that the main subject is present in the full-angle image, the process proceeds to step S104.

[0135] After that, in steps S104 to S109, the same processes as in steps S2 to S7 in Fig. 5 are executed, thereby setting the photographing angle of view (full angle of view).

[0136] In step S110, similarly to the process in step S8 in Fig. 5, low-speed control of the PTZ camera 113 is performed so that the imaging angle of view becomes the set angle. That is, since the information processing system 401 is provided with only the PTZ camera 113 as an imaging device and the images captured by the PTZ camera 113 are always used, the angle of view of the PTZ camera 113 is changed at a low speed.

[0137] In step S111, the captured image is cut out in the same manner as in step S11 in FIG.

[0138] In step S112, the imaging angle setting unit 481 determines whether a certain time has passed since the previous wide-angle imaging. If it is determined that the certain time has not yet passed since the previous wide-angle imaging, the process returns to step S102. Then, the process from step S102 onward is executed.

[0139] On the other hand, if it is determined in step S112 that the certain time has passed since the previous wide-field imaging, the process proceeds to step S113.

[0140] Furthermore, if it is determined in step S103 that the main subject is not captured in the full-angle image captured by the PTZ camera 113, i.e., if it is determined that the main subject does not exist in the full-angle image, steps S104 to S112 are skipped and the process proceeds to step S113.

[0141] In step S113, the information processing unit 451 performs low-speed control of the PTZ camera 113 so that the angle of view becomes wide. Specifically, the imaging angle of view setting unit 481 sets the imaging angle of view to a wide angle of view, and supplies information on the set imaging angle of view to the angle of view control unit 273. The angle of view control unit 273 drives the PTZ of the PTZ camera 113 at a slow speed, similar to the processing of step S9 in Fig. 5, and matches the angle of view of the PTZ camera 113 to the set imaging angle of view (wide angle of view).

[0142] Thereafter, the process returns to step S102, and the processes from step S102 onwards are executed.

[0143] In this manner, the PTZ method and the cropping method are effectively combined using only one PTZ camera 113, making it possible to obtain a captured image with an appropriate angle of view and image quality.

[0144] <<4. Modifications>> Below, a modification of the above-described embodiment of the present technology will be described.

[0145] <Modifications regarding the configuration of the information processing system> For example, in the information processing system 101 in Fig. 2, it is possible to provide two or more wide-angle cameras 111. It is also possible to provide two or more PTZ cameras 113. This increases the number of imaging devices, but makes it possible to obtain images with more types of angles of view at one time.

[0146] 10, it is also possible to provide two or more PTZ cameras 113. This increases the number of image capturing devices, but makes it possible to obtain images with more types of angles of view at once.

[0147] Furthermore, for example, in the information processing system 101 and the information processing system 401, two or more microphones 112 can be provided.

[0148] Also, for example, in the information processing system 101, some or all of the functions of the information processing device 114 (particularly, the functions of the information processing unit 251 in FIG. 4) can be provided in the wide-angle camera 111, the PTZ camera 113, or the switcher 115.

[0149] For example, the wide-angle camera 111 may perform a process of recognizing the situation in the shooting space. For example, the PTZ camera 113 may set the field angle candidates and the shooting field angle, and control the field angle. Furthermore, for example, the PTZ camera 113 may also perform a process of recognizing the situation in the shooting space. Also, for example, the switcher 115 may have all the functions of the information processing device 114.

[0150] Furthermore, for example, in the information processing system 401, a part or all of the functions of the information processing device 114 (particularly, the functions of the information processing unit 451 in FIG. 11) can be provided in the PTZ camera 113 or the switcher 115.

[0151] For example, the PTZ camera 113 may perform a process of recognizing the situation of the shooting space. Also, for example, the PTZ camera 113 may set the field angle candidates and the shooting field angle, and control the field angle. Furthermore, for example, the switcher 115 may have all the functions of the information processing device 114.

[0152] Furthermore, for example, the information processing device 114 may be applied to a server in cloud computing or the like, and the server may provide a service for controlling the angle of view of the PTZ camera 113 .

[0153] <Modification of the method for setting the main subject> For example, it is possible to recognize two or more subjects as the main subject.

[0154] Also, for example, the subject recognition result may be displayed on a display provided in the output unit 207 of the information processing device 114, and the user may use the input unit 206 to select the main subject.

[0155] Furthermore, in the process of step S103 in FIG. 12, a subject that is not captured in the full-angle image may be recognized based on information other than sound (for example, sensor data other than the full-angle image).

[0156] <Modifications regarding setting of angle of view candidates> For example, if an image captured by the wide-angle camera 111 or an image cut out from that captured image is in good condition, candidate angles of view similar to the angle of view of that image may be excluded.

[0157] Furthermore, for example, in studio photography or the like, when the position of the subject is substantially fixed, the user may set candidate angles of view in advance.

[0158] Furthermore, for example, a wide-angle image may be displayed on a display provided in the output unit 207 of the information processing device 114, a user may input a candidate angle of view using the input unit 206, and the candidate angle of view setting unit 281 may set the candidate angle of view based on the user input.

[0159] <Modifications regarding the setting of the photographing angle of view> For example, the display included in the output unit 207 of the information processing apparatus 114 may display the set angle-of-view candidates, and the user may select the necessary angle-of-view candidate using the input unit 206. Then, the shooting angle setting unit 282 or the shooting angle setting unit 481 may set the shooting angle including the angle-of-view candidate selected by the user. Also, in the same manner, the user may be allowed to select the main angle-of-view candidates.

[0160] Furthermore, for example, the shooting angle setting unit 282, the shooting angle setting unit 481, or the user may set the shooting angle by selecting a plurality of main angle-of-view candidates so that the cut-out sizes of all the main angle-of-view candidates are equal to or greater than the threshold value.

[0161] <Modification Example Regarding Cropping of Shooting Image> For example, the cropping unit 263 may estimate the angle-of-view candidate that is most likely to be used from among the angle-of-view candidates included in the shooting angle, and crop the image of the angle corresponding to the estimated angle-of-view candidate.

[0162] Also, for example, the cropping unit 263 may estimate the angle-of-view that is likely to be used within the entire-angle image regardless of the angle-of-view candidates, and crop the image of the estimated angle-of-view.

[0163] Furthermore, for example, the display included in the output unit 207 of the information processing apparatus 114 may display the entire-angle captured shooting image and the angles corresponding to the respective angle-of-view candidates, and the user may select the necessary angle using the input unit 206 so that the image of the selected angle is cropped.

[0164] <Modification Example Regarding PTZ Control of PTZ Camera 113> To accurately control the PTZ of the PTZ camera 113, it is desirable to calculate the PTZ values (pan angle, tilt angle, and zoom magnification) of the PTZ camera 113 using the three-dimensional information including the position in the depth direction of the subject.

[0165] For example, in the first embodiment, three-dimensional information can be obtained by providing two or more wide-angle cameras 111 and using a triangulation technique.

[0166] Furthermore, in the case where there is one wide-angle camera 111, for example, a camera equipped with a depth sensor or a camera equipped with a phase-contrast image can be used as the wide-angle camera 111 to obtain three-dimensional information. Note that the depth sensor method is not particularly limited, and a ToF (Time of Flight) method, a Structured Light method, or the like can be used.

[0167] Furthermore, when there is one wide-angle camera 111, a method can be employed in which the position of each subject in the depth direction is estimated based on a wide-angle image using a technique such as deep learning.

[0168] Also, for example, in the first embodiment, when the wide-angle camera 111 and the PTZ camera 113 are installed in approximately the same position, it is possible to directly estimate the PTZ value of the PTZ camera 113 based on the two-dimensional wide-angle image captured by the wide-angle camera 111.

[0169] Furthermore, for example, a sensor (e.g., a UWB (Ultra Wide Band) sensor, an infrared sensor, etc.) may be attached to a person or other person who may be a subject, and the position of each subject in the depth direction may be recognized based on data from the sensor.

[0170] It should be noted that even if the position of each subject in the depth direction is not recognized, it is possible to speed up the PTZ control of the PTZ camera 113 by utilizing geometric constraints using epipolar lines, for example.

[0171] <Other Modifications> In the second embodiment, the timing for setting the wide field of view angle does not necessarily have to be at regular time intervals, and may be, for example, when a predetermined condition is satisfied.

[0172] In addition, the present technology can be applied to, for example, a case where a shooting space where no special event is taking place is shot, for example, a case where an outdoor landscape is shot, or a case where a predetermined monitoring area is shot, for example.

[0173] <<5.Other>> <Computer configuration example> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, capable of executing various functions by installing various programs.

[0174] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0175] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.

[0176] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0177] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0178] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0179] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0180] <Examples of configuration combinations> The present technology can also be configured as follows.

[0181] (1) a view angle candidate setting unit that sets a plurality of view angle candidates from a photographic image range that can be photographed by the first photographing device for a predetermined photographing space; a photographing angle of view setting unit that sets a photographing angle of view of the first photographing device based on the plurality of candidate angles of view; An information processing device comprising: (2) A recognition unit that recognizes a situation of the photography space based on a photographed image of the photography space. In addition, The view angle candidate setting unit sets the view angle candidate based on a situation of the shooting space. The information processing device according to (1). (3) The recognition unit recognizes, as the situation of the shooting space, a main subject that is a main subject in the shooting space and a related subject that is related to the main subject; The view angle candidate setting unit sets the view angle candidate based on the recognized main subject and the related subject. The information processing device according to (2). (4) The recognition unit recognizes the content of a scene in the shooting space as the situation of the shooting space, The view angle candidate setting unit further sets the view angle candidate based on the content of the scene. The information processing device according to (3). (5) The recognition unit recognizes a situation of the shooting space based on a photographed image photographed by the first photographing device, The imaging angle of view setting unit sets the imaging angle of view to a wide-field angle having the widest field of view at a predetermined timing. The information processing device according to any one of (2) to (4). (6) The imaging angle of view setting unit sets the imaging angle of view to the wide field of view when a main subject that is a main subject in the imaging space does not exist in the image captured by the first imaging device. The information processing device according to (5). (7) The recognition unit recognizes a situation of the photographing space based on a photographed image photographed by a second photographing device. The information processing device according to any one of (2) to (4). (8) The imaging angle of view setting unit sets the imaging angle of view so as to include the plurality of angle of view candidates. 8. The information processing device according to any one of (1) to (7). (9) The imaging angle of view setting unit selects a main angle of view candidate to be included in the imaging angle of view such that a resolution of the main angle of view candidate estimated to be important among the plurality of angle of view candidates is equal to or greater than a predetermined threshold. The information processing device according to (8). (10) A recognition unit that recognizes a situation of the photography space based on a photographed image of the photography space. In addition, The view angle candidate setting unit selects the main view angle candidate from the plurality of view angle candidates based on at least one of a situation of the shooting space and a purpose of the captured image captured by the first image capturing device. The information processing device according to (9). (11) The imaging angle of view setting unit sets a rectangular area including the plurality of angle of view candidates and a margin in at least one direction as the imaging angle of view. The information processing device according to any one of (8) to (10). (12) a cutout unit that cuts out an image having an angle of view corresponding to at least one of the candidate angles of view from an image captured by the first image capture device; The information processing device according to any one of (8) to (11) further comprises: (13) The cutout unit cuts out images having a plurality of angles of view corresponding to the respective candidate angles of view from the captured image captured by the first image capture device. The information processing device according to (12). (14) a view angle control unit that controls pan, tilt, and zoom of the first image capturing device so that the image capturing angle becomes the set view angle; The information processing device according to any one of (1) to (13) further comprises: (15) The angle-of-view control unit controls driving speeds of pan, tilt, and zoom of the first image capturing device based on whether a predetermined role is assigned to an image captured by the first image capturing device. The information processing device according to (14). (16) The first imaging device is provided The information processing device according to any one of (1) to (15). (17) An information processing device, A plurality of candidate angles of view are set from a range of images that can be captured by the imaging device for a given imaging space; A photographing angle of view of the photographing device is set based on the plurality of candidate angles of view. Information processing methods. (18) A plurality of candidate angles of view are set from a range of images that can be captured by the imaging device for a given imaging space; A photographing angle of view of the photographing device is set based on the plurality of candidate angles of view. A program that causes a computer to execute a process.

[0182] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained. [Explanation of symbols]

[0183] 101 information processing system, 111 wide-angle camera, 112 microphone, 113 PTZ camera, 114 information processing device, 115 switcher, 201 CPU, 251 information processing section, 261 calibration section, 262 shooting control section, 263 cropping section, 271 recognition section, 272 angle of view setting section, 273 angle of view control section, 281 angle of view candidate setting section, 282 shooting angle of view setting section, 401 information processing system, 451 information processing section, 461 shooting control section, 471 recognition section, 472 angle of view control section, 481 shooting angle of view setting section

Claims

1. a view angle candidate setting unit that sets a plurality of view angle candidates from a photographic image range that can be photographed by the first photographing device for a predetermined photographing space; a photographing angle of view setting unit that sets a photographing angle of view of the first photographing device based on the plurality of candidate angles of view; a view angle control unit that controls pan, tilt, and zoom of the first photographing device so that the photographing view angle becomes the set view angle, and controls driving speeds of pan, tilt, and zoom of the first photographing device based on whether a predetermined role is assigned to a photographed image photographed by the first photographing device; An information processing device comprising:

2. A recognition unit that recognizes a situation of the photography space based on a photographed image of the photography space. Further preparation, The view angle candidate setting unit sets the view angle candidate based on a situation of the shooting space. The information processing device according to claim 1 .

3. The recognition unit recognizes, as the situation of the shooting space, a main subject that is a main subject in the shooting space and a related subject that is related to the main subject; The view angle candidate setting unit sets the view angle candidate based on the recognized main subject and the related subject. The information processing device according to claim 2 .

4. The recognition unit recognizes the content of a scene in the shooting space as the situation of the shooting space, The view angle candidate setting unit further sets the view angle candidate based on the content of the scene. The information processing device according to claim 3 .

5. The recognition unit recognizes a situation of the shooting space based on a photographed image photographed by the first photographing device, The imaging angle of view setting unit sets the imaging angle of view to a wide-field angle having the widest field of view at a predetermined timing. The information processing device according to claim 2 .

6. The imaging angle of view setting unit sets the imaging angle of view to the wide field of view when a main subject that is a main subject in the imaging space does not exist in an image captured by the first imaging device. The information processing device according to claim 5 .

7. The recognition unit recognizes a situation of the photographing space based on a photographed image photographed by a second photographing device. The information processing device according to claim 2 .

8. The imaging angle of view setting unit sets the imaging angle of view so as to include the plurality of angle of view candidates. The information processing device according to claim 1 .

9. The imaging angle of view setting unit selects a main angle of view candidate to be included in the imaging angle of view such that a resolution of the main angle of view candidate estimated to be important among the plurality of angle of view candidates is equal to or greater than a predetermined threshold. The information processing device according to claim 8.

10. A recognition unit that recognizes a situation of the photography space based on a photographed image of the photography space. Further preparation, The view angle candidate setting unit selects the main view angle candidate from the plurality of view angle candidates based on at least one of a situation of the shooting space and a purpose of the captured image captured by the first image capture device. The information processing device according to claim 9.

11. The imaging angle of view setting unit sets a rectangular area including the plurality of angle of view candidates and a margin in at least one direction as the imaging angle of view. The information processing device according to claim 8.

12. a cutout unit that cuts out an image having an angle of view corresponding to at least one of the candidate angles of view from an image captured by the first image capture device; The information processing device according to claim 8 .

13. The cutout unit cuts out images having a plurality of angles of view corresponding to the respective candidate angles of view from the captured image captured by the first image capture device. The information processing device according to claim 12.

14. The first imaging device is provided The information processing device according to claim 1 .

15. An information processing device, Setting a plurality of candidate angles of view from a range of captured images that can be captured by an image capture device for a predetermined image capture space; setting a photographing angle of view of the photographing device based on the plurality of candidate angles of view; Controlling pan, tilt, and zoom of the imaging device so that the imaging angle of view becomes the set angle; Controlling the driving speeds of pan, tilt and zoom of the image capturing device based on whether a predetermined role is assigned to the image captured by the image capturing device; An information processing method comprising:

16. Setting a plurality of candidate angles of view from a range of captured images that can be captured by an image capture device for a predetermined image capture space; setting a photographing angle of view of the photographing device based on the plurality of candidate angles of view; Controlling pan, tilt, and zoom of the imaging device so that the imaging angle of view becomes the set angle; Controlling the driving speeds of pan, tilt and zoom of the image capturing device based on whether a predetermined role is assigned to the image captured by the image capturing device; A program for causing a computer to execute a process including the steps of:

Images