Information processing device, information processing method, and program

Abstract

An information processing device according to the present technology comprises: a composition selecting unit that selects a camera composition on the basis of a composition selection table in which weight information is associated with each of combinations of an object to be imaged by a camera and a camera composition type; and a composition switching control unit that performs a control for switching the camera composition to a composition selected by the composition selecting unit.

Description

[Technical Field]

[0001] The present technology relates to an information processing device, a method thereof, and a program, and in particular to an imaging control technology. [Background technology]

[0002] Possible control related to image capture includes, for example, controlling the orientation of the camera so that it tracks a detected subject (i.e., subject tracking control), or controlling the zoom of the camera so that it matches a preset angle of view.

[0003] The following Patent Document 1 discloses a technology for controlling the orientation of a camera so that an object is captured. Specifically, Patent Document 1 discloses a technology for detecting the position of a subject based on a captured image and also detecting the position of the subject using a wireless tag carried by the subject, and controlling the orientation of the camera based on these two detection results. Furthermore, Patent Document 2 below discloses a technique for controlling the imaging direction and imaging operation of a camera based on position information and audio information transmitted from an individual information transmitter attached to a subject. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-288745 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-277845 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, for example, when capturing image content for an event such as a live music concert, it is desirable to create high-quality content that does not tire of the viewer, for example by appropriately switching the camera composition. However, selecting an appropriate composition can be difficult for inexperienced users, and the task may require the assistance of an experienced user. Furthermore, manually selecting a composition increases the cost of creating captured image content.

[0006] The present technology has been made in view of the above circumstances, and aims to achieve both improvement in content quality and reduction in work costs related to content creation for captured image content that involves composition switching. [Means for solving the problem]

[0007] The information processing device according to the present technology includes a composition selection unit that selects a composition of the camera based on a composition selection table in which weight information is associated with each combination of a subject to be imaged by the camera and a composition type of the camera, and a composition switching control unit that performs control to switch the composition of the camera to the composition selected by the composition selection unit. In this specification, the term "camera" refers to both a real camera and a virtual camera in which the composition is virtually changed by cutting out a part of an image obtained by the light receiving operation of the real camera. Furthermore, when both real cameras and virtual cameras are defined as "cameras," "capturing an image" refers to the operation of obtaining an image using the camera. According to the above configuration, the composition switching of the camera is automatically performed based on the composition selection table, and the composition switching mode of the camera can be appropriately set by setting the weight information in the composition selection table.

[0008] In addition, an information processing method related to the present technology is an information processing method in which an information processing device selects a composition of the camera based on a composition selection table in which weight information is associated with each combination of a subject to be imaged by the camera and a composition type of the camera, and performs control to switch the composition of the camera to the selected composition. In addition, the program related to the present technology is a program readable by a computer device, and causes the computer device to realize a function of selecting a composition of the camera based on a composition selection table in which weight information is associated with each combination of a subject to be imaged by the camera and a composition type of the camera, and performing control to switch the composition of the camera to the selected composition. These information processing methods and programs realize the information processing device according to the present technology. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image processing system including an information processing device according to a first embodiment of the present technology. [Figure 2] FIG. 1 is an image diagram of a live music venue assumed in the first embodiment. [Figure 3] FIG. 10 is a diagram showing an image captured by a virtual camera. [Figure 4] 5A to 5C are explanatory diagrams of examples of composition types in the first embodiment. [Figure 5] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 6] FIG. 2 is an explanatory diagram of functions of an information processing apparatus according to an embodiment. [Figure 7] FIG. 4 is an explanatory diagram of a composition selection table according to the first embodiment. [Figure 8] 10A and 10B are diagrams for explaining an example of composition selection based on prohibited transition composition information. [Figure 9] 10 is a flowchart illustrating an example of a processing procedure for selecting a composition based on a composition selection table and for performing composition control for switching to the selected composition. [Figure 10] 10 is a flowchart of a process related to updating weight information in the first embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of the configuration of an image processing system according to a second embodiment. [Figure 12]FIG. 10 is an explanatory diagram illustrating an example of the arrangement of the robot cameras in the second embodiment. [Figure 13] FIG. 10 is a diagram showing an example of a composition selection table before weight information is updated. [Figure 14] FIG. 10 is a diagram showing an example of a composition selection table after weight information has been updated. [Figure 15] FIG. 10 is a diagram showing an example of a composition selection table for a robot camera. [Figure 16] 10A and 10B are explanatory diagrams illustrating an example of updating weight information in a composition selection table of a robot camera. [Figure 17] 10 is a flowchart illustrating an example of a specific processing procedure for implementing weight updating according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present technology will be described in the following order with reference to the accompanying drawings. <1. First embodiment> (1-1. Image Processing System as First Embodiment) (1-2. Hardware configuration of information processing device) (1-3. Composition Control as an Embodiment) (1-4. Processing Procedure) 2. Second Embodiment <3. Modifications> <4. Program> <5. Summary of embodiments> <6. This technology>

[0011] <1. First embodiment> (1-1. Image Processing System as First Embodiment) FIG. 1 shows an example of the configuration of an image processing system 100 including an information processing device 1 according to a first embodiment of the present technology. As shown in the figure, the image processing system 100 includes an information processing device 1, a parent camera 2, a child camera 3, a camera platform 4, a switcher 5, and a position detection device 6.

[0012] In this example, the parent camera 2 is singular and the child cameras 3 are plural, and specifically, four child cameras 3 are used. To distinguish between the four child cameras 3, as shown in the figure, a hyphen (-) and a number are added to the end of the code, and the code is written as "3-1," "3-2," "3-3," and "3-4," respectively.

[0013] The parent camera 2 and the child camera 3 are configured as imaging devices that have imaging elements such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor and capture images.

[0014] The pan head 4 is configured as an electronic pan head, and is configured to support the sub camera 3 and to be able to change the orientation of the sub camera 3 in both the pan direction and tilt direction based on an external control signal. In this example, a pan head 4 is provided for each child camera 3, and below, when distinguishing between the four pan heads 4, the symbols are written as "4-1", "4-2", "4-3", and "4-4" with a "- (hyphen)" and a number added to the end of the symbol, as shown.

[0015] In the image processing system 100 , a target is imaged from a plurality of viewpoints by a plurality of cameras serving as a parent camera 2 and a child camera 3 , and a plurality of images obtained based on the image capturing are input to a switcher 5 . The switcher 5 selects and outputs one of the multiple input images based on an operation. In this example, the captured image content for the target event is generated from the images selected and output by the switcher 5. The captured image content generated based on the output image of the switcher 5 can be distributed via a network such as the Internet or transmitted by broadcast waves. Alternatively, the captured image content can be recorded on a predetermined recording medium.

[0016] In this example, the event to be imaged by the cameras is a live music event, and the parent camera 2 and the child camera 3 are installed at the live music venue.

[0017] FIG. 2 is an image diagram of a live music venue assumed in this embodiment. As shown in the figure, a live music venue is equipped with a stage, an audience seating area, and FOH (Front of House). Performers such as musicians and singers perform on the stage. The seating area is located behind the stage and is a space that can accommodate an audience. The FOH is located behind the audience area and is a space where lighting and other elements related to the live performance and various devices for controlling the sound of the venue are located. Personnel from the concert organizers, such as directors and staff, can enter the FOH.

[0018] In this example, the parent camera 2 is a camera that captures the entire stage within its angle of view and is placed at FOH. In this example, the resolution of the image captured by the parent camera 2 is 4K (3840 x 2160), while the resolution of the image output by the switcher 5 is FHD (Full High Definition: 1920 x 1080). Also, in this example, a camera without an optical zoom is used as the parent camera 2. Three of the sub cameras 3 are placed in the space between the stage and the audience area (the space known as the front fence), making it possible to capture the performers on stage within their field of view from a position closer than FOH. As shown in the figure, these three sub cameras 3 are placed in the center in the left-right direction (the direction perpendicular to the front-to-back direction) and at both the left and right ends. The remaining one of the sub cameras 3 is placed at FOH and is used as a camera for capturing a telephoto shot of the performers on stage. In this example, a camera equipped with an optical zoom is used as each of the slave cameras 3. Also, in this example, each of the slave cameras 3 is configured to be able to change the output resolution of the captured image. Specifically, the output resolution can be switched between at least 4K and FHD.

[0019] As will be described later, in the image processing system 100 of this example, the orientation of the sub camera 3 is controlled so as to track a subject such as a performer on stage. For this purpose, the image processing system 100 is provided with a position detection device 6 shown in FIG.

[0020] The position detection device 6 is configured as a device for detecting the position of a wireless tag, and is equipped with multiple receivers 6a that receive radio waves transmitted by the wireless tag. In this example, the position detection device 6 performs position detection using the UWB (Ultra Wide Band) method. By attaching a wireless tag to an object that can be tracked, it becomes possible to detect the position of the object. In this example, the subject to be tracked is a performer on the stage, and in this case, multiple receivers 6a are arranged around the outer periphery of the stage, surrounding the performer's activity area (an area including the center of the stage), as illustrated in Figure 2.

[0021] The method for detecting the subject's position is not limited to the UWB (Ultra Wide Band) method, and various other methods are possible, such as a method using a wireless LAN (Local Area Network), specifically a method in which a plurality of wireless LAN access points are provided and the position is detected based on the difference in arrival time of radio waves between the access points, or a method in which the position is detected using the results of three-dimensional measurement by a ToF (Time of Flight) sensor.

[0022] As shown in FIG. 1, in the image processing system 100 of this example, images from a total of seven systems, CAM1 to CAM7, are input to the switcher 5. For the images CAM4 to CAM7, the images captured by the four sub cameras 3 are input to the switcher 5. The images CAM1 to CAM3 are generated by the information processing device 1 as a computer device based on the images captured by the main camera 2.

[0023] Figure 3 shows images of each of CAM1 to CAM3. The image of CAM1 is basically output as it is, the image captured by parent camera 2. In other words, an image is output with the basic angle of view being the angle of view of the image captured by parent camera 2. With regard to the image of CAM1, electronic zoom processing (i.e., image cropping processing) may be performed on the image captured by parent camera 2, and an image with a narrower angle of view than the angle of view of the image captured by parent camera 2 may be output.

[0024] The images of CAM2 and CAM3 are cut out from the image captured by the parent camera 2. For the images of CAM2 and CAM3, not only the cut-out size but also the cut-out position can be adjusted.

[0025] Here, changing the cutout position for a captured image is equivalent to virtually changing the positional relationship between the camera and the subject, and changing the cutout size for a captured image is equivalent to virtually changing the magnification of the optical zoom. In other words, obtaining an image by changing the cutout position or cutout size for a captured image can be said to be equivalent to changing the composition by moving or operating a virtual camera.

[0026] In this specification, the term "camera" is used as a concept that includes both such a virtual camera and real cameras such as parent camera 2 and child camera 3. In other words, when "camera" is used in this specification, it refers to a concept that includes both a real camera and a virtual camera that virtually changes the composition by cutting out a part of an image obtained by the light receiving operation of the real camera.

[0027] Furthermore, the term "imaging" is used in this specification, and this "imaging" refers to the action of obtaining an image using a camera when both a real camera and a virtual camera are defined as a "camera" as described above.

[0028] In the following description, image clipping may also be referred to as "cutout."

[0029] In this example, the image generated for CAM2 includes all of the subjects, who are performers, detected in the image captured by parent camera 2. In this example, the composition that includes all of the subjects is adaptively changed according to the number and positions of the subjects detected in the image captured by parent camera 2 (a 4K resolution image in this example). In other words, when a new subject is detected in the image captured by parent camera 2, the composition is determined to include that subject, and when the number of detected subjects decreases, the composition is determined based on the positions of the remaining subjects. When no subjects are detected on the stage in the image of CAM2, an FHD-sized image area with the same center as the image captured by parent camera 2 is cropped.

[0030] In this example, the image of CAM3 is generated and output by cutting out an image having a basic angle of view that is narrower than the angle of view of the image captured by the parent camera 2. For example, the image of CAM3 is cut out as an image whose center coincides with the center of the image captured by the parent camera 2.

[0031] Here, the information processing device 1 shown in Figure 1 is configured with, for example, a computer device having a CPU (Central Processing Unit), and generates images from CAM1 to CAM3 as described above based on images captured by the parent camera 2, and also performs composition control for images captured by the child camera 3 as a real camera.

[0032] Composition types employed in this embodiment will be described with reference to FIG. Here, as the composition types, examples of composition types that can be adopted in the composition control of each slave camera 3 will be described. Composition types here include "UP (close-up shot)," "BS (bust shot)," "WS (waist shot)," and "FF (full figure)," as shown in FIGS. 4A to 4D. "UP" is a composition that fits the face of the person being photographed (subject) to the fullest extent of the frame, "BS" is a composition that fits only the part of the person being photographed from the chest to the top of the head within the frame, "WS" is a composition that fits only the part of the person being photographed from the waist to the top of the head within the frame, and "FF" is a composition that fits the entire person being photographed from head to feet within the frame.

[0033] In the composition control of this embodiment, a "composition" is specified by a "composition type" and the type of subject to be imaged (in this example, the performer), as shown in Fig. 4. For example, a composition is specified as capturing a subject to be imaged as a guitar player with a composition type of "WS," or a composition is specified as capturing a subject to be imaged as a vocalist with a composition type of "UP."

[0034] 1 selects a composition for each child camera 3 in accordance with a composition selection table described below for a preset fixed point position and a composition of a tracking target, thereby automatically adjusting the composition without receiving a composition specification operation from a user such as the director or staff member described above. That is, the information processing device 1 of this embodiment selects a composition to be set for each child camera 3 in accordance with the composition selection table in response to the establishment of predetermined conditions, and performs pan and tilt of the corresponding camera platform 4, zoom control of the child camera 3, image cropping, etc. so that the selected composition (fixed point) is set. The composition control according to the embodiment will be described in detail later.

[0035] (1-2. Hardware configuration of information processing device) FIG. 5 is a block diagram showing an example of the hardware configuration of the information processing device 1. As shown in FIG. The information processing device 1 may be in the form of a personal computer, for example. 5, a CPU 11 of an information processing device 1 executes various processes in accordance with a program stored in a nonvolatile memory unit 14 such as a read-only memory (ROM) 12 or an electrically erasable programmable read-only memory (EEP-ROM), or a program loaded from a storage unit 19 into a random access memory (RAM) 13. The RAM 13 also stores data necessary for the CPU 11 to execute various processes, as appropriate. The CPU 11, ROM 12, RAM 13, and nonvolatile memory unit 14 are interconnected via a bus 23. An input / output interface 15 is also connected to this bus 23.

[0036] The input / output interface 15 is connected to an input unit 16 that includes an operator and an operating device. For example, the input unit 16 may be various types of operators or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, or a remote controller. The input unit 16 detects a user operation, and the CPU 11 interprets a signal corresponding to the input operation.

[0037] Furthermore, the input / output interface 15 is connected integrally or separately to a display unit 17 made up of an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) panel, etc., and an audio output unit 18 made up of a speaker, etc. The display unit 17 is a display unit that displays various types of information, and is configured by, for example, a display device provided in the housing of the information processing device 1, a separate display device connected to the information processing device 1, or the like. The display unit 17 displays images for various image processing, moving images to be processed, etc. on the display screen based on instructions from the CPU 11. The display unit 17 also displays various operation menus, icons, messages, etc., that is, GUI (Graphical User Interface), based on instructions from the CPU 11.

[0038] The input / output interface 15 may be connected to a storage unit 19 configured with a hard disk or solid-state memory, or a communication unit 20 configured with a modem or the like. The communication unit 20 performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, and the like.

[0039] A drive 21 is also connected to the input / output interface 15 as required, and a removable recording medium 22 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is appropriately mounted thereon. Drive 21 allows data files such as image files and various computer programs to be read from removable recording medium 22. The read data files are stored in storage unit 19, and images and sounds contained in the data files are output on display unit 17 and audio output unit 18. Computer programs and the like read from removable recording medium 22 are installed in storage unit 19 as needed.

[0040] In this information processing device 1, software can be installed via network communication by the communication unit 20 or via a removable recording medium 22. Alternatively, the software may be stored in advance in the ROM 12, the storage unit 19, or the like.

[0041] (1-3. Composition Control as an Embodiment) Figure 6 is an explanatory diagram of the functions of the CPU 11 of the information processing device 1, and shows the parent camera 2, child camera 3, pan head 4, switcher 5, and position detection device 6 shown in Figure 1, along with functional blocks of the various functions of the CPU 11.

[0042] As shown in the figure, the CPU 11 has functions as a calibration unit F1, a composition selection unit F2, a composition switching control unit F3, a weight update unit F4, an image recognition processing unit F5, an image frame calculation unit F6, a coordinate calculation unit F7, a pan / camera control unit F8, and a cutout image generation unit F9.

[0043] The calibration unit F1 performs a calibration process to find a coordinate transformation matrix for performing coordinate transformation between the coordinate system of the image captured by the parent camera 2 and the coordinate system of the image capture range of each child camera 3. The image capture range of the child camera 3 means the range in which imaging is possible using pan and tilt of the camera platform 4. In this example, the target composition when performing composition control is determined based on the image captured by the parent camera 2. For example, an image frame according to a composition type such as "BS" or "WS" is determined based on the results of image recognition processing performed on the image captured by the parent camera 2. For this reason, the coordinate system of the image captured by the parent camera 2 is used as a master coordinate system, and information on the range of the image frame in the coordinate system of the child camera 3 is obtained by coordinate conversion so that a composition based on the image frame determined in the master coordinate system is realized. The pan head 4 and zoom of the child camera 3 are controlled based on the information on the range of the image frame in the coordinate system of the child camera 3. The calibration unit F1 performs a calibration process to obtain a coordinate transformation matrix for performing this coordinate transformation.

[0044] Specifically, in the calibration process, the display unit 17 displays the images captured by the target child camera 3 and the images captured by the parent camera 2, and the user is prompted to specify positions in each image that show the same position in real space. Based on the information on these specified positions, a coordinate transformation matrix can be found to convert coordinate information in the coordinate system of the parent camera 2 into coordinate information in the coordinate system of the child camera 3. Here, the coordinate transformation matrix is ​​found for each child camera 3. Although not shown in the drawings, an image captured by the target child camera 3 is input to the information processing device 1 during calibration.

[0045] The composition selection unit F2 selects a composition for the camera based on a composition selection table.

[0046] FIG. 7 is an explanatory diagram of the composition selection table. The composition selection table is information in which weight information is associated with each combination of an object to be captured by the camera and a composition type of the camera. Here, an example of a composition selection table is shown for a case where the event to be imaged is a live music event, and the selectable subjects to be imaged are the vocalist, guitar player, or bass player of a music band. The weight information associated with each combination of an object to be imaged and a composition type, in other words, the weight information associated with each composition identified by the combination of an object to be imaged and a composition type, can be said to be information indicating the selection rate of that composition.

[0047] In this embodiment, a composition selection table is prepared for at least each child camera 3, and the composition selection unit F2 selects a composition for each child camera 3 using the corresponding composition selection table. In this example, the composition selection table is stored in a predetermined memory device such as the storage unit 19, and the composition selection unit F2 selects a composition for each of the sub cameras 3 based on the composition selection table thus stored.

[0048] It is conceivable to prepare multiple types of composition selection tables, each with a different combination of composition type and weight information, for each camera targeted for composition control. For example, a composition selection table may be prepared for each of music producers A, B, and C, each with a different combination of composition type and weight information to facilitate the selection of compositions frequently used by those music producers. The user may then select from these composition selection tables the table to be used for composition selection by the composition selection unit F2. This allows the user to select which music producer's style the captured image content will be finished in.

[0049] Here, in this example, the composition selection unit F2 selects a composition for each camera based on prohibited transition composition information so as to prevent a composition transition that is a predetermined prohibited transition from occurring. The prohibited transitions referred to here refer to composition transitions that are prohibited from occurring between images that are sequentially selected by the switcher 5 .

[0050] The composition selection unit F2 selects a composition based on the composition selection table based on the above-mentioned prohibited transition composition information and the selection history information of the camera by the switcher 5 (corresponding to composition history information) so that the composition transition between images selected sequentially by the switcher 5 does not become a prohibited transition defined in the prohibited transition composition information. Specifically, based on the prohibited transition composition information and the composition of the camera currently selected by the switcher 5, the composition selection unit F2 identifies as a "prohibited transition composition" from among the compositions in the composition selection table of each child camera 3 those that would result in a prohibited transition if next selected by the switcher 5, and performs composition selection based on the weight information in the composition selection table, targeting only compositions excluding these prohibited transition compositions.

[0051] An example of composition selection based on such prohibited transition composition information will be described with reference to FIG. Specifically, Figure 8 illustrates an example of composition selection for a certain child camera 3 when the composition of the child camera 3 selected by the switcher 5 is "Base WS," and transitions from the "Base WS" composition to the "Base UP" composition and transitions to a composition with the "Base" as the imaging subject are defined as prohibited transitions. In this case, in the composition selection table, the "Bass UP" composition and the composition with the "Bass" as the imaging subject are prohibited transition compositions, so the composition selection unit F2 selects compositions excluding these compositions (in the illustrated example, the composition with the vocalist as the imaging subject, the "Bass WS" composition, and the "Bass FF" composition). This prevents the composition transition of the output image by the switcher 5 from becoming a prohibited transition even if the target child camera 3 is switched to the selected composition and the image captured by the child camera 3 is selected by the switcher 5.

[0052] In this example, the composition selection unit F2 performs composition selection based on the composition selection table for each child camera 3 described above, on the condition that image selection has been performed by the switcher 5. In other words, when the switcher 5 newly selects one of the captured images from each camera, a composition is selected for each child camera 3 based on the composition selection table, and the composition of each child camera 3 is controlled to the selected composition.

[0053] Furthermore, in this example, the composition selection unit F2 selects a composition based on the composition selection table, excluding cameras for which images are being selected by the switcher 5. As described above, in this example, composition selection is performed based on the composition selection table on the condition that image selection has been performed by the switcher 5, but in this case, composition selection is performed based on the composition selection table, excluding cameras for which images are being selected by the image selection. As described above, by excluding a camera for which an image is being selected by the switcher 5 from the targets for composition selection, it is possible to prevent composition switching from being performed for a camera selected by the switcher 5 even though an image captured by the camera is being selected.

[0054] Furthermore, the composition selection unit F2 in this example selects a composition based on the composition selection table, targeting only the musical accompaniment section identified from the audio analysis result of the event to be imaged. As an example, each slave camera 3 is equipped with a microphone, and audio data picked up by the microphone is attached to the image data captured by each slave camera 3. The composition selection unit F2 then performs audio analysis on the attached audio data to determine whether or not the section is a musical accompaniment section. Based on the result of this determination, the composition selection unit F2 selects a composition based on a composition selection table, targeting only the musical accompaniment section. The method for acquiring audio data is not limited to the above-described method using a camera microphone. For example, audio data collected by a microphone provided in a device other than a camera, such as a PA (Public Address) device, may also be used for audio analysis. In this case, the audio data is not attached to the image captured by the camera, but is treated as data independent of the captured image. The determination of whether or not a section is an audio accompaniment section can also be made based on an operational input from the user.

[0055] In the captured image content of a music event, there is less need for composition changes in the intervals between songs, such as the MC (Master of Ceremonies) portion, compared to the intervals during songs. By selecting the composition only for the musical accompaniment section as described above, it is possible to prevent unnecessary composition switching from occurring between songs, thereby reducing the processing load associated with composition switching.

[0056] 6, the composition switching control unit F3 performs control to switch the camera composition to the composition selected by the composition selection unit F2. Specifically, the composition switching control unit F3 in this example performs processing to instruct the image frame calculation unit F6, which will be described later, on the composition selected by the composition selection unit F2 for each child camera 3.

[0057] The weight update unit F4 updates the weight information in the composition selection table. As an example, the weight update unit F4 updates the weight information based on selection history information of the camera by the switcher 5. In this case, every time a camera is selected by the switcher 5, in other words, a captured image is selected, the weight update unit F4 performs processing to store information about the captured image and its composition as selection history information in a predetermined storage device such as the storage unit 19, and updates the weight information in the composition selection table based on the selection history information.

[0058] In this example, the weight update unit F4 performs processing to increase the weight of the weight information for compositions that are frequently selected by the switcher 5 (selection frequency is equal to or greater than a certain frequency) among the weight information for each composition in each composition selection table. As a result, the weights are updated so that the switcher 5 is more likely to select compositions that are popular and often used. Therefore, it is possible to control the composition of the captured image content so as to match the preferences of the user of the switcher 5 as closely as possible, thereby improving the quality of the captured image content.

[0059] Furthermore, the weight update unit F4 updates the weights based on the content of the event to be captured. Specifically, in this example, when a solo part of an instrument, such as a guitar solo, is detected as the content of the event to be captured, the weight update unit F4 updates the weight information so as to increase the weight of a composition in which the player of that instrument is the image target. In this example, the detection of the solo part of the instrument is performed based on the results of audio analysis of the event to be imaged, for example, based on the results of audio analysis of the audio data attached to the images captured by each sub camera 3 as described above, or audio data obtained by PA equipment, etc. The detection of the solo part of the instrument can also be performed based on an operational input by the user.

[0060] As described above, by increasing the weight of the composition that targets the player of an instrument in response to the detection of the solo part of that instrument, it becomes easier to select an appropriate composition that corresponds to the content of the event, thereby improving the quality of the captured image content.

[0061] Here, with regard to the weight update unit F4, there are at least two possible examples of timing for updating the weight information. The first example is an example in which the weight information is updated on the condition that an image has been selected by the switcher 5. This makes it possible to adjust which compositions are more likely (or less likely) to be selected for other cameras that were not selected, in response to the selection of a certain camera (certain composition) by the switcher 5. Therefore, it is possible to make it easier for a composition suitable for the next image selection by the switcher 5 to be selected for a camera that has not been selected, and it is possible to increase the possibility that the captured images to be selected by the switcher 5 will include captured images with an appropriate composition, leading to an improvement in the quality of the captured image content.

[0062] In the second example, the weighting information is updated when a predetermined sound change is detected from the audio analysis results of the event to be imaged, such as when a predetermined melody change is detected, such as when a vocal part is changed to a solo instrument part. This makes it possible to update the weights so that, when the sound content of the event to be imaged transitions to a specific content, it becomes easier to select an appropriate composition that matches the specific content, for example, when a sound change that is estimated to have transitioned to a guitar solo part is detected, the weights are updated so that a composition in which the guitar player is the imaged subject is more likely to be selected. Therefore, the quality of the captured image content can be improved.

[0063] The image recognition processing unit F5 performs image recognition processing on the image captured by the parent camera 2. The image recognition processing here includes at least processing to recognize the face, position, and range of the subject as a person, and bone estimation for the subject recognized as a person (for example, generation of a simple model of the human body that represents the structure of the human body with main parts such as the face, torso, arms, and legs).

[0064] In this example, information on the position of a subject as an imaging target is calculated comprehensively using not only the detection result by the position detection device 6 but also the subject recognition result based on the image captured by the parent camera 2. That is, when calculating an image frame for tracking a certain subject, the image frame calculation unit F6 described below calculates information on the position of the subject based on the position detection result by the position detection device 6 and the subject recognition result by the image recognition processing unit F5.

[0065] The image frame calculation unit F6 calculates an image frame to realize the composition instructed by the composition switching control unit F3. As will be understood from the previous explanation, the image frame referred to here is an image frame in the coordinate system of the image captured by the parent camera 2. In this example, the target of composition control is each child camera 3, and the composition switching control unit F3 instructs a composition for each child camera 3, so the image frame calculation unit F6 calculates an image frame for each child camera 3 for the instructed composition.

[0066] As a specific example, if a composition is instructed for a certain child camera 3 in which the subject to be captured is "vocals" and the composition type is "up," the image frame calculation unit F6 acquires position information of the subject as "vocals" based on the position detection result by the position detection device 6 and the subject recognition result by the image recognition processing unit F5. At the same time, bone estimation information for the subject as "vocals" is required to realize an "up" composition, so this bone estimation information is acquired from the image recognition processing unit F5. Then, based on this acquired information, an image frame for capturing the "vocals" at the size of "up" is calculated.

[0067] Furthermore, the image frame calculation unit F6 also calculates the image frames for the virtual cameras (CAM1 to CAM3 in this example) as needed, for example, based on an operational input from the user.

[0068] The coordinate calculation unit F7 converts the image frame information (coordinate information in the coordinate system of the parent camera 2) for the child cameras 3 (CAM4 to CAM7 in this example) calculated by the image frame calculation unit F6 into coordinate information in the coordinate system of the child cameras 3. For this coordinate conversion, the coordinate conversion matrix determined by the calibration unit F1 is used.

[0069] The pan / tilt head / camera control unit F8 controls the pan and tilt of the pan / tilt head 4 and the zoom of the sub-camera 3 as necessary for each sub-camera 3, based on the information of the image frame that has been coordinate converted by the coordinate calculation unit F7, so as to obtain a composition that captures the range indicated by the information of the image frame. This makes it possible to switch the composition of each slave camera 3 to the composition selected by the composition selection unit F2.

[0070] The cutout image generation unit F9 cuts out the image captured by the parent camera 2 as necessary according to the image frame information (coordinate information in the coordinate system of the parent camera 2) for the virtual camera calculated by the image frame calculation unit F6, and generates captured images of CAM1, CAM2, and CAM3.

[0071] (1-4. Processing Procedure) Next, with reference to the flowcharts of FIGS. 9 and 10, a specific example of a processing procedure for realizing the composition control according to the first embodiment described above will be described. The processes shown in FIGS. 9 and 10 are executed by the CPU 11 of the information processing device 1.

[0072] FIG. 9 shows an example of a processing procedure for selecting a composition based on a composition selection table and for controlling composition switching to the selected composition. First, in step S101, the CPU 11 waits until a composition switching trigger is generated. As will be understood from the above explanation, in this example, the condition for generating the composition switching trigger is that an image is selected by the switcher 5 and that the selected image is in a musical accompaniment section, and in step S101 the CPU 11 performs processing to wait until the conditions are met.

[0073] When a composition switching trigger occurs, the CPU 11 proceeds to step S102 and performs processing to identify a composition that will result in a prohibited transition for each target camera. That is, based on the prohibited transition composition information described above and information on the composition of the camera currently selected by the switcher 5 (based on the selection history information of the camera described above), processing is performed to identify, as a "prohibited transition composition," a composition that will result in a prohibited transition if selected next by the switcher 5 from among the compositions in the composition selection table of each slave camera 3. For confirmation, in the specification process of step S102, there may be a case where a specification result is obtained that there is no composition that corresponds to the prohibited transition composition in at least one of the composition selection tables.

[0074] In step S103 following step S102, the CPU 11 performs a process of selecting a composition based on the composition selection table for compositions excluding compositions that result in prohibited transitions. That is, for a composition selection table that has a composition that corresponds to a prohibited transition composition, composition selection is performed based on weight information for compositions excluding the prohibited transition composition. For a composition selection table that does not have a composition that corresponds to a prohibited transition composition, composition selection is performed based on weight information for each composition in the table. If all compositions are prohibited transition compositions, the composition is determined in anticipation of the next composition selection with respect to the selection of other cameras.

[0075] Here, as described above, when composition selection is performed based on the composition selection table excluding a camera for which an image is being selected by the switcher 5, composition selection based on the composition selection table is not performed for the relevant camera in step S103. This makes it possible to prevent composition switching from being performed for a camera for which an image is being selected by the switcher 5.

[0076] In step S104 following step S103, the CPU 11 controls the composition of the target camera to achieve the selected composition. That is, the child camera 3 for which composition selection was performed based on the composition selection table in step S103 is set as the target camera, and the composition of the target camera is switched to the selected composition by performing processing as the composition switching control unit F3, image recognition processing unit F5, image frame calculation unit F6, coordinate calculation unit F7, and pan / tilt head / camera control unit F8 described above.

[0077] In step S105 following step S104, the CPU 11 determines whether a processing termination condition is met. The processing termination condition here is a predetermined condition that has been set in advance as a condition for terminating the series of processing shown in Fig. 9, such as when the generation of captured image content has reached a state where it is necessary to terminate the generation of captured image content.

[0078] If it is determined that the processing termination condition is not met, the CPU 11 returns to step S101. As a result, in response to the occurrence of another composition switching trigger, composition selection based on the composition selection table and switching to the selected composition are executed again for the corresponding child camera 3.

[0079] On the other hand, if it is determined that the processing end condition is met, the CPU 11 ends the series of processing shown in FIG.

[0080] FIG. 10 is a flowchart of a process related to updating weight information. In step S201, the CPU 11 waits for a weight update trigger to occur. As will be understood from the above explanation, the weight update trigger can be one of the first and second examples described above. In the first example, in step S201, the CPU 11 waits for an image to be selected by the switcher 5. In the second example, in step S201, the CPU 11 waits for a predetermined sound change to be detected from the audio analysis results of the event to be imaged (for example, a change from a singing part to a solo instrument part to be detected).

[0081] If a weight update trigger occurs, the CPU 11 proceeds to step S202 and performs processing to determine the weight for each composition in the composition selection table for each camera. For example, as explained above, when updating the weight so that compositions that are frequently selected by the switcher 5 are more likely to be selected, a higher value is determined for the weight information of compositions that are selected by the switcher 5 at a certain frequency or more. Furthermore, when updating the weighting based on the content of the event to be imaged, for example, in response to a case where a solo part of an instrument is detected, a higher value is determined for the weighting information of the composition in which the player of that instrument is the imaged subject.

[0082] In step S203 following step S202, the CPU 11 performs a process of updating the weights to the determined weights. That is, the CPU 11 performs a process of updating the numerical values ​​of the corresponding weight information among the weight information in the composition selection table for each camera to the numerical values ​​determined in step S202.

[0083] In step S204 following step S203, the CPU 11 determines whether a processing termination condition is met. The processing termination condition here is a predetermined condition that is set in advance as a condition for terminating the series of processing shown in Fig. 10, such as when the generation of captured image content has reached a state where it is necessary to terminate the generation of captured image content.

[0084] If it is determined that the processing end condition is not met, the CPU 11 returns to step S201, whereby the corresponding weight information is updated in response to the occurrence of a weight update trigger again.

[0085] On the other hand, if it is determined that the processing end condition is met, the CPU 11 ends the series of processing shown in FIG.

[0086] In the above, an example was given in which composition selection based on the composition selection table is performed on the condition that image selection has been performed by the switcher 5, but composition selection based on the composition selection table can also be performed in response to changes in the content of the event to be imaged. For example, based on the results of audio analysis of the event being imaged, a composition may be selected based on a composition selection table on the condition that a predetermined change in melody (change in event content), such as a change from a singing part to a solo instrument part, is detected. In this way, by selecting a composition based on the composition selection table in response to changes in the content of the event to be imaged, it is possible to ensure that an appropriate composition change is performed in response to changes in the content of the event to be imaged.

[0087] Furthermore, composition selection based on the composition selection table may be performed depending on the amount of time that has elapsed since the composition was switched based on the previous composition selection. For example, composition selection may be performed on the condition that a certain amount of time has elapsed since the composition was switched based on the previous composition selection.

[0088] It is also conceivable that composition selection based on the composition selection table may be performed in response to a predetermined operation input by the user. In this case, for example, by operating a predetermined button, composition selection based on the composition selection table and switching to the selected composition are performed for cameras other than the camera currently selecting an image by the switcher 5.

[0089] Furthermore, in updating the weight information, it is also possible to lower the weight of compositions that have been selected by the switcher 5 based on the selection history information of the camera by the switcher 5. This allows weights to be updated so that compositions that have been selected by the switcher 5 are less likely to be selected, thereby preventing the same compositions from appearing frequently in captured image content and preventing a decline in content quality.

[0090] 2. Second Embodiment Next, a second embodiment will be described. In the second embodiment, weight information is updated based on input information relating to an imaging target from an external device. In the following description, parts that are the same as parts that have already been described will be given the same reference numerals and description thereof will be omitted.

[0091] FIG. 11 is a diagram showing an example of the configuration of an image processing system 100A according to the second embodiment. The differences from the image processing system 100 shown in Figure 1 are that an information processing device 1A is provided instead of the information processing device 1, that a server device 7 is provided that is capable of communicating with the information processing device 1A via a network NT, and that a robot camera 8 is added. In this example, the network NT is the Internet.

[0092] The server device 7 collectively refers to a server device that serves as a distribution server for captured image content generated based on the selected output image by the switcher 5, and a server device that serves as an SNS (Social Networking Service) server. The SNS server here refers to a server device that has at least the function of causing information processing terminals such as smartphones, tablet terminals, and personal computers connected to the network NT to acquire web page data of an SNS site, and the function of accepting information input such as posted information from the information processing terminals and reflecting the information on the SNS site.

[0093] In this example, the function of the captured image content distribution server includes a function of accepting tips from information processing terminals.

[0094] The hardware configuration of the computer device that constitutes the server device 7 is the same as that shown in FIG. 5, and therefore a description using an illustration will be omitted. Here, it is also possible to adopt a configuration in which the functions of the server device 7 are realized by the cooperation of a plurality of computer devices.

[0095] The robot camera 8 has a camera unit that captures images using an imaging element such as a CCD image sensor or a CMOS image sensor, and a support unit that supports the camera unit, and the support unit is configured to be self-propelled. Although detailed illustrations are omitted, the robot camera 8 of this example is configured so that the position of the camera unit in the height direction and the imaging direction can be freely adjusted by panning and tilting. The captured image obtained by the camera unit of the robot camera 8 is input to the switcher 5 (CAM8 in the figure). Furthermore, adjustment of the height position and imaging direction of the camera unit of the robot camera 8, and control of the movement thereof are performed under the control of the information processing device 1A.

[0096] FIG. 12 is an explanatory diagram of an example of the arrangement of the robot camera 8. In FIG. As shown in the figure, in this example, the robot camera 8 is placed on a stage at an event venue (in this example, a live music venue). In this case, the travel control of the robot camera 8 is performed based on information about a predetermined travel line La on the stage. In this example, a travel line La is determined for each of the performers (e.g., each member of an idol group) positioned on the stage, and when switching to a composition in which a certain performer is the subject of imaging, the information processing device 1A controls the travel of the robot camera 8 so that the robot camera 8 travels on the travel line La determined for that performer.

[0097] Although not shown in the figure, the CPU 11 in the information processing device 1A also has functions related to composition control for the child camera 3, including the calibration unit F1, composition selection unit F2, composition switching control unit F3, weight update unit F4, image recognition processing unit F5, image frame calculation unit F6, coordinate calculation unit F7, and pan / tilt head / camera control unit F8 shown in Figure 6 above. However, the second embodiment differs from the first embodiment in that the weight update unit F4 has the functions described below.

[0098] In this case, the weight update unit F4 has a function of updating the weight information based on input information related to the imaging target from an external device. Specifically, the weight update unit F4 in this case updates the weight information in the composition selection table based on the tip information from the information processing terminal to the server device 7 or the posted information from the information processing terminal to the SNS site. More specifically, the weight information in the composition selection table is updated so that a composition that captures the member who has donated the most amount of tips or posted the most comments is more likely to be selected.

[0099] Here, posted information such as tips and supportive comments for performers at an event can be said to be information (viewer evaluation information) regarding viewer evaluations of the performers of the event being filmed. Viewer evaluations here refer to evaluations of the performers by viewers of the captured image content obtained by filming the event being filmed. The weight update according to the tips given to performers or posted information as described above can be rephrased as a weight update based on viewer evaluation information.

[0100] A specific example of updating the weight information in this case will be described with reference to FIGS. FIG. 13 shows an example of the composition selection table before the weight information is updated, and FIG. 14 shows an example of the composition selection table after the weight information is updated.

[0101] Here, we show an example in which member A has the highest amount of tips and the highest number of comments posted among a group of three people, member A to member C. As shown by the transition from Figure 13 to Figure 14, the weights for all compositions in which members other than member A are photographed are set to 0, and only compositions in which member A is photographed are given a weight greater than 0. This ensures that when a composition is selected based on the composition selection table, the composition is switched to one that captures the member who has donated the most tips or posted the most comments.

[0102] In addition, in the second embodiment, for the robot camera 8 shown in FIG. 11, composition selection and switching control to the selected composition are performed based on a composition selection table, but the composition selection table for the robot camera 8 may be, for example, a table such as that shown in FIG. 15. Specifically, the composition selection table for the robot camera 8 can be a table that includes information on the running line La for each composition type. As shown in the figure, for a composition in which member A is the subject of image capture, information on the running line A, which is the running line La corresponding to member A, is stored as the composition type information, and for a composition in which member B is the subject of image capture, information on the running line B, which is the running line La corresponding to member B, is stored as the composition type information. In this case, the composition type information stores information indicating the running line La determined for the member to be captured.

[0103] In this example, angle information is included in the composition type information in the composition selection table for the robot camera 8. In the example of Fig. 15, for some compositions of each member, angle information as a low angle (an angle looking up at the subject from below) is stored together with information on the running line La as composition type information. Note that angle information is not limited to low angles, but can take a variety of forms, such as an angle at which the person being photographed is viewed from eye level (which, for the viewer of the content, is an angle at which they are looking at the person being photographed).

[0104] In the second embodiment, the weight of the robot camera 8 is also updated according to the amount of tips and the number of posted comments. Specifically, in this case, the weight update unit F4 also updates the weight information in the composition selection table of the robot camera 8 so that a composition that captures the member who has donated the most amount of tips or the member who has posted the most comments is more likely to be selected.

[0105] FIG. 16 is an explanatory diagram of an example of updating weight information in the composition selection table of the robot camera 8. In FIG. Here, we show an example of an update when member A has the largest amount of tips or the largest number of comments posted between member A and member B. As can be seen by comparing with Figure 15, in this case, the weights of all compositions for member B are set to 0, and the weights of the compositions for member A that combine "driving line A and UP" and "driving line A and low angle" are each updated to values ​​greater than 0. Specifically, in this case, as shown in the figure, the weight of the composition that combines "driving line A and UP" is set to 20, and the weight of the composition that combines "driving line A and low angle" is set to 80, so that the weights of compositions that include low angles are the greatest.

[0106] By updating the weights as described above, it is possible to ensure that the robot camera 8 always switches to a composition that captures the member who has donated the most tips or posted the most comments. Furthermore, by updating the weights as described above, it is possible to make it most likely that a composition capturing the relevant member at a low angle will be selected.

[0107] As shown in Figure 15, in this example, the original weight for low-angle compositions is set to 0, and when the weight is updated according to the amount of tips or the number of posted comments, the weight for low-angle compositions becomes the highest. This makes it possible to ensure that the low-angle composition is only revealed when the composition is switched according to the amount of tipping or the number of posted comments.

[0108] FIG. 17 is a flowchart showing a specific example of a processing procedure for implementing the weight update according to the second embodiment described above. The process shown in FIG. 17 is executed by the CPU 11 in the information processing device 1A.

[0109] In this case, the CPU 11 waits until the weight update condition based on the external information is satisfied in step S301. Specifically, in this example, the CPU 11 waits for the end of the period for accepting tips or comments posted to members. It is also assumed that the acceptance of posted comments may be performed in advance before the distribution of the captured image content. In this case, the process of step S301 may be a process of waiting for the arrival of any timing between the end of the acceptance of posted comments and the start of a song that is the target of composition switching according to the number of posted comments.

[0110] If the condition for updating the weights based on the external information is met, the CPU 11 proceeds to step S302 and determines the weights based on the external information. Specifically, as explained above with reference to Figures 13 to 16, the composition selection table of each child camera 3 and the composition selection table of the robot camera 8 are used to determine the numerical value of the weight information so that a composition that captures the member with the largest amount of tips or the largest number of posted comments is more likely to be selected. At this time, the weights in the composition selection table of the robot camera 8 are determined so that a low-angle composition is most likely to be selected, as described above.

[0111] In step S303 following step S302, the CPU 11 performs processing to update the weights to the determined weights. That is, processing is performed to update the weight information values ​​for each composition in the composition selection table of each child camera 3 and the composition selection table of the robot camera 8 to the values ​​determined in step S302.

[0112] In this case, in response to executing the processing of step S303, the CPU 11 determines in step S204 whether the processing termination condition is met, and if it determines that the processing termination condition is not met, it returns to step S301, and if it determines that the processing termination condition is met, it ends the series of processing shown in Figure 17.

[0113] In the above, we have given an example of updating the weights so that a composition that captures the performer with the largest amount of tips or the largest number of comments posted is more likely to be selected, but it is also possible to set the composition selection table and update the weights so that a composition that captures the top multiple performers with the largest amount of tips or the largest number of comments posted is more likely to be selected.

[0114] In addition, weight updates according to the amount of tips and the number of comments posted could be performed so that differences in the amount of tips and the number of comments posted by each performer are reflected as differences in weight information for each performer in the composition selection table.

[0115] <3. Modifications> Here, the embodiment is not limited to the specific examples described above, and various modified configurations can be adopted. For example, in the above example, composition selection and composition switching using a composition selection table are performed for the composition of a real camera, but composition selection and composition switching using a composition selection table can also be performed for the composition of a virtual camera.

[0116] In addition, in the above example, a camera being selected by the switcher 5 is excluded from the composition selection based on the composition selection table. However, it is also possible to exclude a camera for which less than a certain time has elapsed since the last composition change from the composition selection based on the composition selection table.

[0117] Furthermore, with regard to updating the weights in the composition selection table, it is also conceivable to automatically update the weights for compositions that have not been selected by the switcher 5 when a weight update start condition is met or even after a certain period of time has elapsed since the previous weight update timing. For example, it is conceivable to reduce the weight for the relevant composition so that it becomes less likely to be selected. Alternatively, conversely, it is conceivable to increase the weight for the relevant composition so that it becomes more likely to be selected.

[0118] Furthermore, in the above example, the information processing device 1 (or 1A) selects a composition based on a composition selection table, but it is also possible to configure the camera to have the composition selection table and select a composition based on the composition selection table.

[0119] In addition, although the above example shows an example in which a device that selects a composition based on a composition selection table and a device that controls a composition changing means such as the camera platform 4 so as to switch to the selected composition are configured as an integrated unit, these devices can also be configured separately. In this case, it is conceivable that the latter device is placed at the event venue, and the former device is placed in a location separate from the event venue, for example as a cloud server, so as to be able to communicate with the latter device via a network.

[0120] Furthermore, in the above example, a composition is selected based on the composition selection table, excluding the camera currently selected by the switcher 5 (the camera currently outputting a program program). However, if the switcher 5 is capable of selecting a so-called NEXT (Preview) camera (the camera that will output a program program next after the camera currently outputting a program program: a candidate camera for program program output), it is also possible to select a composition based on the composition selection table, excluding both the camera currently outputting a program program and the NEXT (Preview) camera, or only the NEXT (Preview) camera.

[0121] Furthermore, in the above description, an example has been given in which the switcher 5 is configured as a hardware device, but the switcher 5 may also be realized by a software program executed by an information processing device.

[0122] Furthermore, with regard to composition switching control, if a composition other than the prohibited transition composition has already been set as the composition of the target camera and that composition has not yet been output as a program, the composition of the target camera may be maintained without being switched, regardless of the weight information in the composition selection table of the target camera.

[0123] Furthermore, in the explanation so far, an example has been given of the event to be imaged being a live music event, but this technology can also be suitably applied to other events to be imaged, including events such as musicals where the performances are held on a stage (either indoors or outdoors), program recordings in a studio, and sporting events such as baseball, soccer, basketball, and volleyball.

[0124] <4. Program> The information processing device (1 or 1A) has been described as an embodiment, and the program of the embodiment is a program that causes a computer device such as a CPU to execute processing as the information processing device 1 or information processing device 1A.

[0125] The program of the embodiment is a program that can be read by a computer device and causes the computer device to realize the function of selecting a camera composition based on a composition selection table in which weight information is associated with each combination of a subject to be captured by the camera and a camera composition type, and performing control to switch the camera composition to the selected composition. That is, this program corresponds to a program that causes a computer device to execute the processing described with reference to FIG. 9 and the like.

[0126] Such a program can be stored in advance in a computer-readable recording medium, such as a ROM, HDD (Hard Disk Drive), or SSD (Solid State Drive). Alternatively, the program can be temporarily or permanently stored in a removable recording medium such as a semiconductor memory, a memory card, an optical disk, a magneto-optical disk, or a magnetic disk. Such removable recording medium can also be provided as a so-called package software. In addition, such a program can be installed onto a personal computer or the like from a removable recording medium, or can be downloaded from a download site to a required information processing device such as a smartphone via a network such as a LAN or the Internet.

[0127] <5. Summary of embodiments> As described above, the information processing device (1 or 1A) as an embodiment includes a composition selection unit (F2) that selects a composition for the camera based on a composition selection table in which weight information is associated with each combination of a subject to be imaged by the camera and a composition type of the camera, and a composition switching control unit (F3) that performs control to switch the composition of the camera to the composition selected by the composition selection unit. According to the above configuration, the composition switching of the camera is automatically performed based on the composition selection table, and the composition switching mode of the camera can be appropriately set by setting the weight information in the composition selection table. Therefore, for captured image content involving composition switching, it is possible to achieve both an improvement in content quality and a reduction in the work costs involved in creating the content.

[0128] In addition, in the information processing device as an embodiment, there are multiple cameras, and an image is selected from the images captured by the multiple cameras by a switcher (same as above 5), and the composition selection unit performs composition selection for the camera based on the composition selection table on the condition that an image has been selected by the switcher. This makes it possible, when images captured by multiple cameras are selected by a switcher to generate captured image content, to switch the composition of other cameras that were not selected in response to the selection of a certain camera (certain composition) by the switcher. Therefore, it is possible to switch the composition of the camera that was not selected to a composition suitable for the next selection by the switcher, and it is possible to improve the quality of the content while reducing the work cost by automating the composition selection.

[0129] Furthermore, in the information processing device according to the embodiment, the composition selection unit selects a composition for the camera based on the composition selection table in response to changes in the content of the event to be captured by the camera. This makes it possible to switch the camera's composition in response to changes in the content of the target event, such as a change in the melody of the music being played when the target event is a live music event. Therefore, it is possible to switch the composition appropriately in response to changes in the content of the event to be imaged.

[0130] Furthermore, in the information processing device according to the embodiment, the composition type includes the type of angle of view. This allows the camera composition to be switched to compositions with different angles of view, such as BS (bust shot) and WS (waist shot).

[0131] In the information processing device according to the embodiment, the composition type includes the angle type (see FIGS. 7 and 15, etc.). This allows for camera composition switching to change the direction of the line of sight relative to the subject, such as an angle looking up at the subject from below (low angle) or an angle looking from eye level.

[0132] Furthermore, the information processing device according to the embodiment includes a weight update unit (F4) that updates the weights in the composition selection table. By updating the weights in the composition selection table, it is possible to change the ease with which each composition is selected. Therefore, for example, in a guitar solo part, the weights are updated so that a composition in which the guitar is the subject of image capture is more likely to be selected, and it is possible to switch compositions appropriately according to the situation.

[0133] Furthermore, in the information processing device of the embodiment, there are multiple cameras, and a switcher selects images from among the images captured by the multiple cameras, and the weight update unit updates the weights based on the camera selection history information by the switcher. This makes it possible to, for example, update the weights so that compositions that have been frequently selected by the switcher in the past are more likely to be selected, thereby making it easier to select compositions that are preferred and frequently used by the switcher, or conversely, update the weights so that compositions that have been selected by the switcher in the past are less likely to be selected, thereby preventing the same compositions from appearing frequently in captured image content, i.e., preventing a decline in the quality of the content. Therefore, as composition switching based on the composition selection table, appropriate composition switching can be performed based on the past camera selection history (composition selection history) by the switcher, thereby improving the quality of the content.

[0134] In the information processing device according to the embodiment, the weight update unit increases the weight of a composition that has been selected by the switcher. As a result, the weights are updated so that compositions that are preferred and frequently used in the switcher are more likely to be selected. Therefore, it is possible to control the composition of the captured image content so as to match the preferences of the switcher user as closely as possible, thereby improving the quality of the captured image content.

[0135] Furthermore, in the information processing device according to the embodiment, the weight update unit updates the weight based on the content of the event captured by the camera. This makes it possible to easily select an appropriate composition according to the content of the event to be photographed, for example, when the event to be photographed is a guitar solo part at a live music concert, it becomes easier to select a composition in which the guitar is the subject of photography. Therefore, the quality of the captured image content can be improved.

[0136] Furthermore, in the information processing device according to the embodiment, the weight update unit updates the weight based on the audio analysis result of the imaging target event. This makes it possible to update the weights so that an appropriate composition is more easily selected depending on the content of the event estimated from the sound aspect, such as the guitar solo part if the event to be imaged is a live music concert. Therefore, the quality of the captured image content can be improved.

[0137] In addition, in the information processing device as the embodiment, the weight update unit updates the weight based on input information relating to the imaging target from an external device (see the second embodiment). This makes it possible to update the weights based on input information about the subject from an external device, such as tips and supportive comments about the subject as a particular member of an idol group, etc. For example, the weights can be updated so that a composition that captures the subject that has received the most tips and supportive comments is more likely to be selected. Therefore, it is possible to generate captured image content with a composition that appropriately reflects the intention of the person who inputs information about the subject of image capture, such as the recipient of the content.

[0138] In the information processing device according to the embodiment, the weight update unit updates the weight based on viewer evaluation information relating to viewer evaluations of performers of an event that is imaged by the camera. The viewer evaluation here refers to the evaluation of the performers by the viewers of the captured image content obtained by capturing the event to be captured. Examples of viewer evaluation information include tips and supportive comments for the performers. With the above configuration, it is possible to update the weights based on viewer evaluation information for a performer, such as tips and supportive comments for a specific member of an idol group. For example, the weights can be updated so that a composition capturing the performer who has received the most tips and supportive comments is more likely to be selected. Therefore, it is possible to generate captured image content with a composition that appropriately reflects the intentions of the content viewer.

[0139] Furthermore, in the information processing device of the embodiment, there are multiple cameras, and an image is selected from images captured by the multiple cameras using a switcher, and the weight update unit updates the weights on the condition that an image has been selected by the switcher. This makes it possible to adjust which compositions are made more likely (or less likely) to be selected for other cameras that were not selected, in response to the selection of a certain camera (certain composition) by the switcher. Therefore, it is possible to make it easier for a camera that has not been selected to select a composition that is suitable for the next image selection by the switcher, and it is possible to increase the possibility that the captured images selected by the switcher will include captured images with an appropriate composition, thereby improving the quality of the captured image content.

[0140] Furthermore, in the information processing device according to the embodiment, the weight update unit updates the weight on the condition that a predetermined sound change is detected from the audio analysis result of the event to be imaged by the camera. This makes it possible to update the weights so that a composition in which a guitar is the subject of imaging is more likely to be selected when a sound change that is estimated to indicate that the event to be imaged has transitioned to a guitar solo part at a live music concert is detected, for example.In other words, when the sound content of the event to be imaged transitions to a specific content, the weights can be updated so that an appropriate composition that matches the specific content is more likely to be selected. Therefore, the quality of the captured image content can be improved.

[0141] In addition, in the information processing device of the embodiment, there are multiple cameras, and a switcher selects images from among the images captured by the multiple cameras, and the composition selection unit selects compositions based on the composition selection table, excluding the camera for which images are being selected by the switcher. This makes it possible to prevent the composition from being switched for the camera selected by the switcher even though the captured image is currently being selected.

[0142] Furthermore, in the information processing device according to the embodiment, the composition selection unit selects a composition based on the composition selection table only for the musical accompaniment section identified from the audio analysis result of the event to be imaged by the camera. In the case of captured image content of a music event, there is less need for composition changes in inter-song portions such as MC portions compared to portions during songs. According to the above configuration, it is possible to prevent unnecessary composition switching even between songs, and to reduce the processing load related to composition switching.

[0143] Furthermore, in the information processing device of the embodiment, there are multiple cameras, and a switcher selects images from among the images captured by the multiple cameras, and the composition selection unit selects compositions based on the camera selection history information by the switcher and the composition selection table. This makes it possible to prevent switching to a composition that is prohibited when a composition transition between specific compositions is defined as a prohibited transition, such as switching from a composition in which a guitar is the subject of imaging to a composition in which a bass is the subject of imaging.

[0144] In addition, an information processing method as an embodiment is an information processing method in which an information processing device selects a camera composition based on a composition selection table in which weight information is associated with each combination of a subject to be captured by the camera and a camera composition type, and performs control to switch the camera composition to the selected composition. This information processing method can also provide the same functions and effects as the information processing device of the above embodiment.

[0145] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0146] <6. This technology> The present technology can also be configured as follows. (1) a composition selection unit that selects a composition of the camera based on a composition selection table in which weight information is associated with each combination of an object to be photographed by the camera and a composition type of the camera; a composition switching control unit that performs control to switch the composition of the camera to the composition selected by the composition selection unit. Information processing device. (2) There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The composition selection unit executes composition selection for the camera based on the composition selection table on the condition that an image selection has been performed by the switcher. The information processing device according to (1) above. (3) The composition selection unit selects a composition for the camera based on the composition selection table in response to a change in the content of an event to be captured by the camera. The information processing device according to (1) or (2). (4) The composition type includes the type of angle of view. The information processing device according to any one of (1) to (3). (5) The composition type includes the angle type. The information processing device according to any one of (1) to (3). (6) a weight update unit that updates the weights in the composition selection table; The information processing device according to any one of (1) to (5). (7) There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The weight update unit updates the weight based on selection history information of the camera by the switcher. The information processing device according to (6) above. (8) The weight update unit increases the weight of a composition that has been selected by the switcher. The information processing device according to (7) above. (9) The weight update unit updates the weight based on the content of the event to be captured by the camera. The information processing device according to any one of (6) to (8). (10) The weight update unit updates the weight based on a result of audio analysis of the image capture target event. The information processing device according to (9) above. (11) The weight update unit updates the weight based on input information about the imaging target from an external device. The information processing device according to any one of (6) to (10). (12) The weight update unit updates the weight based on viewer evaluation information regarding viewer evaluations of performers of the event to be imaged by the camera. The information processing device according to any one of (6) to (11). (13) There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The weight update unit updates the weights on the condition that an image is selected by the switcher. The information processing device according to any one of (6) to (12). (14) The weight update unit updates the weight on the condition that a predetermined sound change is detected from a result of sound analysis of the event to be imaged by the camera. The information processing device according to any one of (6) to (13). (15) There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The composition selection unit selects a composition based on the composition selection table, excluding the camera for which an image is being selected by the switcher. The information processing device according to any one of (1) to (14). (16) The composition selection unit selects a composition based on the composition selection table only for a musical accompaniment section identified from a result of audio analysis of an event to be imaged by the camera. The information processing device according to any one of (1) to (15). (17) There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The composition selection unit selects a composition based on the selection history information of the camera by the switcher and the composition selection table. The information processing device according to any one of (1) to (16). (18) The information processing device A composition of the camera is selected based on a composition selection table in which weight information is associated with each combination of an object to be photographed by the camera and a composition type of the camera, and control is performed to switch the composition of the camera to the selected composition. Information processing methods. (19) A computer readable program, The computer device realizes a function of selecting a composition of the camera based on a composition selection table in which weight information is associated with each combination of an object to be photographed by the camera and a composition type of the camera, and performing control to switch the composition of the camera to the selected composition. program. [Explanation of symbols]

[0147] 100,100A Image Processing System 1,1A Information processing equipment 2 Parent Camera 3, 3-1, 3-2, 3-3, 3-4 Child Camera 4,4-1,4-2,4-3,4-4 Head 5 Switcher 6 Position detection device 6a receiver 7 Server equipment 8. Robot Camera 11 CPU 12 ROM 13 RAM 14 Non-volatile memory section 20 Communications Department 23 Bus F1 Calibration Section F2 Composition selection section F3 Composition switching control section F4 Weight update section F5 Image recognition processing unit F6 Image frame calculation section F7 Coordinate calculation section F8 Head and Camera Control Unit F9 Cutout image generator NT Network La driving line

Claims

1. a composition selection unit that selects a composition of the camera based on a composition selection table in which weight information is associated with each combination of an object to be photographed by the camera and a composition type of the camera; a composition switching control unit that performs control to switch the composition of the camera to the composition selected by the composition selection unit. Information processing device.

2. There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The composition selection unit executes composition selection for the camera based on the composition selection table on the condition that an image selection has been performed by the switcher. The information processing device according to claim 1 .

3. The composition selection unit selects a composition for the camera based on the composition selection table in response to a change in the content of an event to be captured by the camera. The information processing device according to claim 1 .

4. The composition type includes the type of angle of view. The information processing device according to claim 1 .

5. The composition type includes the angle type. The information processing device according to claim 1 .

6. a weight update unit that updates the weights in the composition selection table; The information processing device according to claim 1 .

7. There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The weight update unit updates the weight based on selection history information of the camera by the switcher. The information processing device according to claim 6 .

8. The weight update unit increases the weight of a composition that has been selected by the switcher. The information processing device according to claim 7 .

9. The weight update unit updates the weight based on the content of the event to be captured by the camera. The information processing device according to claim 6 .

10. The weight update unit updates the weight based on a result of audio analysis of the image capture target event. The information processing device according to claim 9 .

11. The weight update unit updates the weight based on input information about the imaging target from an external device. The information processing device according to claim 6 .

12. The weight update unit updates the weight based on viewer evaluation information regarding viewer evaluations of performers of the event to be imaged by the camera. The information processing device according to claim 6 .

13. There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The weight update unit updates the weights on the condition that an image is selected by the switcher. The information processing device according to claim 6 .

14. The weight update unit updates the weight on the condition that a predetermined sound change is detected from a result of sound analysis of the event to be imaged by the camera. The information processing device according to claim 6 .

15. There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The composition selection unit selects a composition based on the composition selection table, excluding the camera for which an image is being selected by the switcher. The information processing device according to claim 1 .

16. The composition selection unit selects a composition based on the composition selection table only for a musical accompaniment section identified from a result of audio analysis of an event to be imaged by the camera. The information processing device according to claim 1 .

17. There are a plurality of cameras, and an image is selected from the images captured by the plurality of cameras by a switcher; The composition selection unit selects a composition based on the selection history information of the camera by the switcher and the composition selection table. The information processing device according to claim 1 .

18. The information processing device A composition of the camera is selected based on a composition selection table in which weight information is associated with each combination of an object to be photographed by the camera and a composition type of the camera, and control is performed to switch the composition of the camera to the selected composition. Information processing methods.

19. A computer readable program, The computer device realizes a function of selecting a composition of the camera based on a composition selection table in which weight information is associated with each combination of an object to be photographed by the camera and a composition type of the camera, and performing control to switch the composition of the camera to the selected composition. program.

Images