Information processing device, information processing method, and program

Abstract

An information processing device according to one embodiment of the present technology comprises an acquisition unit, a detection unit, and a control unit. The acquisition unit acquires image-capture information acquired from one or a plurality of image-capturing units for capturing content. The detection unit detects feature information indicating a feature of the content, on the basis of the image-capture information. The control unit controls each image-capturing unit on the basis of the feature information. Thus, various framing control can be executed in accordance with various situations including an object to be captured.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program that can be applied to photography and the like.

[0002] Patent Document 1 describes an imaging system that captures images of multiple subjects using multiple imaging devices that can communicate with each other. In this imaging system, when a first subject and a second subject are close to each other, the system determines a composition so that the first subject and the second subject fit within the imaging range. This allows for a variety of imaging controls (see, for example, paragraphs

[0024] to

[0043] ,

[0049] to

[0062] , and Figures 1, 2, 5, and 6 of the specification of Patent Document 1).

[0003] JP 2011-211598 A

[0004] There is a demand for an information processing device, an information processing method, and a program that can perform a variety of framing controls in accordance with such various situations including the object being photographed.

[0005] In view of the above circumstances, an object of the present technology is to provide an information processing device, an information processing method, and a program that are capable of executing various framing controls.

[0006] To achieve the above object, an information processing device according to one embodiment of the present technology includes an acquisition unit, a detection unit, and a control unit. The acquisition unit acquires shooting information acquired from one or more shooting units that shoot content. The detection unit detects feature information indicating features of the content based on the shooting information. The control unit controls each of the shooting units based on the feature information.

[0007] In this information processing device, shooting information is acquired from one or more shooting units that shoot content, feature information indicating the features of the content is detected based on the shooting information, and each of the shooting units is controlled based on the feature information, thereby enabling a variety of framing controls to be performed.

[0008] The feature information may include a target object related to the content. In this case, the control unit may control the imaging unit to perform a framing operation on the target object related to the content.

[0009] The target object may include a character related to the content. In this case, the control unit may control at least one of the imaging units to perform a framing operation on the character.

[0010] The feature information may include at least one of the situation of the content or the actions of the characters.

[0011] The framing operation may include a plurality of patterns, in which case the control unit may switch the pattern of the framing operation based on the feature information.

[0012] The framing operation may include zooming and tracking the target object.

[0013] The information processing device may further include a setting unit that performs initial settings related to shooting of the content, which becomes the feature information. In this case, the setting unit may present setting information on a user interface that allows the initial settings to be set.

[0014] The initial settings may include the number of the image capturing units, a person to be captured in the content, and a time for framing the person to be captured.

[0015] When a person to be photographed is set, the control unit may control at least one of the photographing units to perform a framing operation on the person to be photographed.

[0016] The initial setting may include a positional relationship of the image capturing unit. In this case, the setting unit may present, on the user interface, a position of the image capturing unit that is suitable for the content.

[0017] The initial setting may include an allocation of time for framing a person who is a subject of the content and another person who is not a subject of the content, in which case the control unit may perform control to perform the framing operation based on the allocation of time.

[0018] The content may include sports or music.

[0019] An information processing method according to one aspect of the present technology is an information processing method executed by a computer system, the information processing method including acquiring shooting information acquired from one or more shooting units that shoot content, detecting feature information indicating features of the content based on the shooting information, and controlling each of the shooting units based on the feature information.

[0020] A recording medium having a program recorded thereon according to an embodiment of the present technology causes a computer system to execute the following steps: acquiring shooting information acquired from one or more shooting units that shoot content; detecting feature information indicating features of the content based on the shooting information; and controlling each of the shooting units based on the feature information.

[0021] 1 is a diagram schematically illustrating a photography system according to a first embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the information processing device shown in FIG. 1. FIG. 3 is a flowchart illustrating a framing operation. FIG. 4 is a flowchart illustrating a framing operation in the case of a right-handed batter. FIG. 5 is a flowchart illustrating a framing operation in the case of a left-handed batter. FIG. 6 is a diagram illustrating an example of a framing operation for each camera in the case of three cameras. FIG. 7 is a schematic diagram illustrating the arrangement of cameras in the case of two cameras. FIG. 8 is a diagram illustrating an example of a framing operation for each camera in the case of two cameras. FIG. 9 is a schematic diagram illustrating the arrangement of cameras in the case of one camera. FIG. 10 is a diagram illustrating an example of a framing operation for each camera in the case of one camera. FIG. 11 is a schematic diagram illustrating the arrangement of cameras in the case of content according to a second embodiment. FIG. 12 is a diagram illustrating a flowchart of a framing operation in soccer. FIG. 13 is a diagram illustrating an example of a framing operation for each camera in the case of three cameras in soccer. FIG. 14 is a schematic diagram illustrating the arrangement of cameras in the case of two cameras. FIG. 10 is a diagram showing an example of framing operations for each camera in the case of two cameras in soccer.

[0022] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0023] First Embodiment FIG. 1 is a diagram schematically illustrating an imaging system 100 according to a first embodiment of the present technology.

[0024] 1, the photography system 100 includes a photography unit 1, an information processing device 10, and a user device 5. In the first embodiment, a situation in which photography is being performed with the photography system 100 will be described, with a baseball game as an example of content.

[0025] The imaging system 100 is not limited to baseball, but can also be applied to sports, particularly goal sports such as soccer, basketball, rugby, American football, handball, hockey, water polo, etc. Furthermore, the imaging system is not limited to games, and may also be used to capture practice scenes such as knocking practice.

[0026] The imaging unit 1 is composed of a camera capable of capturing content and automatically framing a target object, and a support unit such as a pan head or tripod that supports the camera and enables it to rotate in any direction. For example, the imaging unit 1 tracks the target object and controls the pan head in order to frame the target object. In the following description, the imaging unit 1 including the camera and support unit may be simply referred to as a camera.

[0027] Here, framing refers to fitting a target object within the camera's angle of view. In other words, framing refers to various operations such as tracking the target object, zooming in on the target object, and obtaining a bird's-eye view of the entire game including the target object. Similarly, a framing operation refers to transmitting a control signal from the information processing device 10 to control the imaging unit 1 to perform the above-mentioned framing.

[0028] Furthermore, in this embodiment, the target object includes characters related to the content. For example, in the case of baseball, examples include people participating in the content, such as players, managers, and spectators. In addition, the target object also includes objects related to the content. For example, in baseball, examples include a baseball ball, a bat, or bases (first base, second base, etc.), and in soccer, examples include tools used in the game, such as a soccer ball or soccer goal.

[0029] One or more camera units 1 are provided. For example, in Fig. 1, camera 1 is provided to take pictures from behind the backstop, camera 2 is provided to take pictures from the first base side, and camera 3 is provided to take pictures from the third base side.

[0030] The photographing information photographed by the photographing unit 1 is output to the information processing device 10. The photographing information includes not only video, sound, and images, but also information on the positional relationship between the cameras (locations).

[0031] Based on the acquired shooting information, the information processing device 10 outputs a control signal to each camera to perform a framing operation. In this embodiment, the information processing device 10 causes cameras 1 to 3 to perform various framing operations based on the situation of the baseball game.

[0032] The information processing device 10 also edits the video of the match captured by the imaging unit 1. The edited video can be transmitted to the user device 5 or distributed to a terminal owned by another user, a video sharing site, or the like.

[0033] The user device 5 is a terminal owned by the user 6 that activates the photography system 100. For example, the user 6 inputs the number of cameras to be placed into the photography system 100 via the user device 5 and performs an operation to cause the photography system 100 to start photography, thereby photographing a match. The user 6 can also view, from the user device 5, a video of the match that has been photographed and edited.

[0034] The user device 5 is not limited to a specific device and may be a personal computer (PC), a mobile terminal such as a smartphone, or the like. A dedicated application or the like may be installed on the user device 5 to enable the user to operate the photography system 100. Cooperation with the photography unit 1 may also be achieved by any method.

[0035] FIG. 2 is a block diagram showing an example of the configuration of the information processing device 10 shown in FIG.

[0036] As shown in FIG. 2 , the information processing device 10 includes a setting unit 11 , an acquisition unit 12 , a detection unit 13 , a control unit 14 , an editing unit 15 , and a distribution unit 16 .

[0037] The setting unit 11 performs initial settings for the photography system 100. The initial settings include the number of photography units 1 (cameras), the person who will be the focus of the photography (the person who will be the subject of framing), settings related to framing such as the allocation of cut rates within the video, and settings related to editing the video.

[0038] For example, the user 6 sets a person (hereinafter referred to as a favorite player) to be filmed for the content via the user device 5, and the filming unit 1 is controlled so that framing is performed around the favorite player. Specifically, it is possible to set the filming to include 20% more shots of the favorite player compared to normal filming when a favorite player is not set. Also, not setting a favorite player may be included in the initial settings. In this case, the content of the match is filmed fairly without focusing on a specific player.

[0039] The setting unit 11 also presents to the user a suitable location for the imaging unit 1 for the content. For example, the setting unit 11 acquires position information (including relative positional relationships) from each camera and presents to the user a suitable imaging location for capturing the content based on the position information. Examples of the proposed locations include behind the backstop, the first baseman, and the third baseman in the case of baseball (see FIGS. 1 and 7). Examples of the proposed locations include near the halfway line, behind the goal, and in the case of soccer (see FIGS. 12 and 15).

[0040] The presentation method is not limited, and the placement location may be presented via a GUI (Graphical User Interface) or by voice. That is, the GUI image information or voice information that can be confirmed from the user device 5 corresponds to setting information that can set the initial settings presented in the user interface.

[0041] The acquisition unit 12 acquires the shooting information acquired by the shooting unit 1. In this embodiment, when there are multiple shooting units 1, a timestamp beacon is added to the shooting information, and the shooting information of each camera is synchronized based on the timestamp.

[0042] The detection unit 13 detects feature information indicating the features of the content based on the shooting information. For example, if the content is baseball, the feature information may include specific scenes related to various baseball situations, such as the pitcher's pitching motion (such as the pitching posture and the timing of the moment of pitching), the pitcher's attribute information (such as whether the pitcher is right-handed or left-handed), the umpire's decision (ball, strike, out), the catcher's catching, the batter's hitting motion (such as the hitting posture and the timing of the moment of hitting), the batter's attribute information (such as whether the batter is right-handed or left-handed), the baseball, the path of the hit ball, a hit, a home run, an error, a passed ball, a point scored, a change of offense and defense, and a stolen base. Note that the feature information is not limited to the information detected by the detection unit 13 based on the shooting information; the content of the initial settings made by the user as described above also constitutes feature information. Therefore, information such as whether a character is a favorite player is also included in the feature information. Furthermore, if a favorite player is not set in the initial settings, characteristic information indicating that there is no favorite player may be set.

[0043] The feature information may also include information about things outside of baseball (spectators other than players and the manager). For example, the feature information may include applause, cheers, cheers, cheering songs, and signs from the manager. In other words, the feature information can be rephrased as information about the situation of the content, target objects related to the content, or the actions of the characters.

[0044] The detection unit 13 outputs the detected feature information to the control unit 14. Note that the method by which the detection unit 13 detects the feature information is not limited, and for example, an analysis technique of captured video or sound may be used, or the type of feature information to be acquired from the number of cameras and their positions may be detected using AI learning as learning data.

[0045] The control unit 14 controls each of the imaging units 1 based on the characteristic information. In this embodiment, the control unit 14 performs framing operations for each imaging unit 1 based on various characteristic information. Specifically, the control unit 14 switches the camera zoom and camera direction (tripod gimbal control), and performs framing operations to capture wide-angle images that show the entire game, close-up images of players' expressions, the direction in which the baseball flew or was thrown, the moment the baseball was caught, the status of runners on the bases, the state of the spectators, the state of the manager, and the like.

[0046] Information on the framing operation by the control unit 14 is output to the editing unit 15. In addition, shooting information of the image captured by the imaging unit 1 that has been subjected to the framing operation is also output to the editing unit 15.

[0047] The editing unit 15 edits the captured video based on the feature information and framing operations. In this embodiment, the editing unit 15 performs editing such as directing the video, adding information about the content situation, and arranging sounds.

[0048] For example, video production includes slow-motion editing of the moment a batter hits the ball or a home run, cutting the video (determining the angle and composition), etc. Note that the method for cutting the video is not limited, and appropriate cutting may be set based on the number and placement of cameras by a computing device that has undergone AI learning or the like.

[0049] In the case of baseball, adding information about the status of the content includes, for example, a scoreboard, a count such as strikes, updates on the status of runners on base, play-by-play broadcasts, etc. Specifically, a scoreboard or count may be added to the bottom left of the screen, and play-by-play broadcasts such as hits and grounders when players hit the ball, and runners being put out may be added.

[0050] For example, sound placement may involve separating applause, cheers, cheering, cheering songs, and the voices of players and referees from the captured image information and placing those sounds in appropriate locations. Specifically, distant sounds (such as applause from the third base) may be reduced based on the user's expected viewing position (e.g., the first base side). Furthermore, when audio equipment is placed around the user, the sense of realism may be enhanced by playing specific sounds from audio equipment placed behind the user.

[0051] Note that various user instructions may be added to the above editing, such as whether to add a scoreboard or live coverage, or whether to use slow motion effects.

[0052] The distribution unit 16 outputs the edited video to the user device 5, a video distribution site, or other external device. Alternatively, the edited video may be uploaded to a cloud for storage, or may be distributed via the cloud.

[0053] Fig. 3 is a flowchart showing the framing operation. Fig. 4 is a flowchart showing the framing operation for a right-handed batter. Fig. 5 is a flowchart showing the framing operation for a left-handed batter. Fig. 6 is a diagram showing an example of the framing operation for each camera in the case of three cameras.

[0054] 3, in this photography system 100, when a batter enters the batter's box (step 101), the framing operation differs depending on the batter's attribute information. That is, a determination is made as to whether the batter is a right-handed batter or not (step 102). Note that the method of determination is not limited, and whether the batter is a right-handed batter or a left-handed batter may be determined from the video image, or may be determined from the player's profile, etc.

[0055] If the batter is a right-handed batter (YES in step 102), the process proceeds to process A shown in FIG.

[0056] As shown in Fig. 4, when the pitcher throws the ball (step 201), framing operation 1 is executed by control unit 14 (step 202). As shown in Fig. 6, framing operation 1 causes camera 1 to capture a panoramic view, camera 2 to capture a close-up of the batter, and camera 3 to frame the runner. The reason for performing such framing control for multiple cameras is that camera 1 is located in the center of the baseball stadium and is therefore suitable for capturing a panoramic view, and as for cameras 2 and 3, if the batter is a right-handed batter, framing the batter with camera 2 will allow the batter to be captured from the front, making it possible to clearly capture the batter's facial expressions, etc., whereas framing the batter with camera 3 will result in the batter being captured from behind, making it impossible to capture the batter's facial expressions, etc.

[0057] The term "panoramic view" includes framing that shows people involved in the play from pitching to hitting, such as the pitcher, batter, catcher, and umpire. The panoramic view may also include the entire field 20 and the spectator seats.

[0058] If the batter has hit (YES in step 203), it is determined whether a runner has already reached base (step 204). Note that a hit refers to a state in which the batter or runner is able to run the bases, such as a ground ball, fly ball, hit, or home run. In other words, a foul ball does not constitute a hit.

[0059] If a runner is on base (YES in step 204), the control unit 14 executes framing operation 2 (step 205). As shown in Fig. 6, framing operation 2 causes camera 1 to track the ball, camera 2 to track the batter, and camera 3 to track the runner. Note that camera 3 prioritizes tracking a runner who is advancing (for example, a runner on third base over a runner who was on first base).

[0060] It should be noted that zooming may be performed when tracking target objects such as a batter, a runner, a ball, etc. The magnification ratio is not limited, and zooming may be performed to the extent that the facial expressions of the players can be seen.

[0061] If there is no runner on base (NO in step 204), the control unit 14 executes framing operation 3 (step 206). As shown in Figure 6, framing operation 3 causes camera 1 to track the ball, camera 2 to track the batter, and camera 3 to capture a panoramic view centered on the direction in which the ball is traveling.

[0062] After framing operation 2 or framing operation 3 is executed, if the umpire judges the batter to be out or safe (step 207), the process moves to process C.

[0063] If the batter fails to hit the ball (NO in step 203), framing operation 1 continues to be executed (step 208). If the batter fails to hit the ball and strikes out, i.e., the batter is out (YES in step 209), the process moves to process C. If the batter is not out (if it is a ball, a foul, or the batter has not struck out), framing operation 1 is executed again in time with the pitcher's pitch (steps 201 and 202).

[0064] As shown in FIG. 3, when the batter is out, the next batter enters the batter's box or there is a change (switch of offense and defense) (step 103), and the process returns to step 101 again.

[0065] If the batter is a left-handed batter (NO in step 102), the process proceeds to process B.

[0066] As shown in Fig. 5, when the pitcher throws the ball (step 301), framing operation 4 is executed by control unit 14 (step 302). As shown in Fig. 6, framing operation 4 causes camera 1 to capture a full view, camera 2 to frame the runner's movements (enough to cover first to third base), and camera 3 to frame a close-up of the batter. This is because, if the batter is a left-handed batter, framing the batter with camera 3 allows the batter to be photographed from the front, making it possible to clearly capture the batter's facial expressions, etc., whereas framing the batter with camera 2 results in the batter being photographed from behind, making it impossible to capture the batter's facial expressions, etc.

[0067] If the batter has hit the ball (YES in step 303), it is determined whether or not a runner is already on base (step 304).

[0068] If a runner is on base (YES in step 304), the control unit 14 executes framing operation 5 (step 305). As shown in Figure 6, framing operation 5 causes camera 1 to track the ball, camera 2 to track the runner, and camera 3 to track the batter.

[0069] If there is no runner on base (NO in step 304), the control unit 14 executes framing operation 6 (step 306). As shown in Fig. 6, framing operation 6 causes camera 1 to track the ball, camera 2 to take a panoramic shot centered on the direction in which the ball is traveling, and camera 3 to track the batter.

[0070] After framing operation 2 or framing operation 3 is executed, if the umpire judges the batter to be out or safe (step 307), the process moves to process C.

[0071] If the batter fails to hit the ball (NO in step 303), framing operation 4 continues to be executed (step 308). If the batter fails to hit the ball and strikes out, i.e., the batter is out (YES in step 309), the process moves to process C. If the batter is not out, framing operation 4 is executed again in time with the pitcher's pitch (steps 301 and 302).

[0072] Fig. 7 is a schematic diagram showing the locations of two cameras, and Fig. 8 is a diagram showing an example of the framing operation of each camera in the case of two cameras.

[0073] As shown in Figure 7, when there are two cameras, camera 1 is placed on the first side, and camera 2 is placed on the third side.

[0074] 8, in the case of two cameras, framing operations 1 to 6 are executed as follows: Which framing operation is executed follows the flowcharts of FIGS.

[0075] Framing operation 1 causes camera 1 to capture a panoramic view and camera 2 to capture the entire field 20 with the runner at the center. Framing operation 2 causes camera 1 to track the ball and camera 2 to track the runner from a distance. Framing operation 3 causes camera 1 to track the batter and camera 2 to track the ball.

[0076] Framing operation 4 causes camera 1 to capture the entire field 20 with the runner at the center, and camera 2 to capture the entire view. Framing operation 5 causes camera 1 to track the ball, and camera 2 to track the runner from a distance. Framing operation 6 causes camera 1 to track the ball, and camera 2 to track the batter.

[0077] Fig. 9 is a schematic diagram showing the placement of a single camera, and Fig. 10 is a diagram showing an example of the framing operation of each camera when there is only one camera.

[0078] As shown in FIG. 9, for example, in the case of one camera, camera 1 is placed behind the backstop.

[0079] 10, in the case of one camera, framing operations 1 to 6 are executed as follows: Which framing operation is executed follows the flowcharts of FIGS.

[0080] Framing operation 1 causes camera 1 to capture the entire scene, framing operation 2 causes camera 1 to track the runner, and framing operation 3 causes camera 1 to track the ball.

[0081] Framing operation 4 causes camera 1 to capture a full view, framing operation 5 causes camera 1 to track the runner, and framing operation 6 causes camera 1 to track the ball.

[0082] The location of the camera is not limited. For example, if one camera is used and it is desired to observe the catcher's movements, the camera may be placed in a position that provides a view in front of the catcher.

[0083] Here, an example of a framing operation when a user sets a favorite player during initial setup will be described. For example, when a user sets a specific player, such as a son, as the favorite player, the control unit 14 executes the framing operation shown in FIG. 11 . Note that any method may be used to set the favorite player. For example, the information processing device 10 may display a list of facial images of the participating players acquired in advance on the screen of the user device 5, and the user 6 may select a facial image from the list and set it as the facial image of the favorite player. If the information processing device 10 cannot provide facial images of the participating players, the user 6 may upload an image including the face of the favorite player via the user device 5 and provide it to the information processing device 10. Then, image recognition technology may be used on the captured video to determine whether the player is the favorite player.

[0084] 11 is a diagram showing an example of the framing operation for each camera when a favorite player is set and there are three cameras. The framing operation to be performed follows the flowcharts of FIGS. 3 to 5.

[0085] 11, framing operation 1 causes camera 1 to take a panoramic shot, camera 2 to take a close-up shot of the batter, and camera 3 to frame the shot around the favorite player. Framing operation 2 causes camera 1 to track the ball, camera 2 to track the batter, and camera 3 to frame the shot around the favorite player. Framing operation 3 causes camera 1 to track the ball, camera 2 to track the batter, and camera 3 to frame the shot around the favorite player.

[0086] Framing operation 4 causes camera 1 to take a panoramic shot, camera 2 to frame the shot around the favorite player, and camera 3 to take a close-up shot of the batter. Framing operation 5 causes camera 1 to track the ball, camera 2 to frame the shot around the favorite player, and camera 3 to track the batter. Framing operation 6 causes camera 1 to track the ball, camera 2 to frame the shot around the favorite player, and camera 3 to track the batter.

[0087] In this way, by setting a favorite player, a video can be created that focuses on the favorite player. The framing operation for the favorite player may be changed depending on the favorite player's offensive or defensive position (top or bottom). For example, if the favorite player is playing first base, the framing operation may be performed as shown in FIG. 11, and if the favorite player is a batter, the framing operation for camera 2 or camera 3 may be performed with a different location (such as a spectator or another player) as the center.

[0088] Furthermore, if multiple favorite players are set, the framing may be performed according to the time allocation for framing each favorite player set in the initial settings, or each of camera 2 and camera 3 may frame each favorite player.

[0089] The above framing operation is merely an example, and may be set by any method, such as AI learning. For example, any framing operation may be generated based on the number of cameras and their positions. Furthermore, for example, a target object for the framing operation may be set based on a cut that has received high praise from users or the like when rating an edited video (such as a scene where a runner scores home).

[0090] As described above, the information processing device 10 according to the first embodiment acquires shooting information from one or more cameras 1 that shoot baseball games, detects feature information indicating the characteristics of the baseball based on the shooting information, and performs framing operations on each of the cameras 1 based on the feature information. This makes it possible to perform a variety of framing controls.

[0091] In the past, when filming using multiple camera devices, each camera device controlled what object to film and how to film it. For example, when multiple cameras were used for one object, each camera simply tracked the object, but in a sports game such as baseball, there were many points to focus on, and the timing and cutting of filming those points depended heavily on the skill of the cameraman who manually operated the camera.

[0092] This technology detects characteristics of a baseball game (points of interest) from the captured video and audio, and switches framing patterns based on those characteristics. This allows even amateur users to simply set up a camera and shoot footage that is similar to what will be broadcast.

[0093] Second Embodiment A second embodiment according to the present technology will be described. In the following description, descriptions of parts having the same configurations and functions as those of the information processing device 10 described in the above embodiment will be omitted or simplified.

[0094] 12 is a schematic diagram showing the locations of cameras in the case of content according to the second embodiment, in which soccer is taken as an example of content.

[0095] As shown in FIG. 12, camera 1 is placed near the halfway line 21 of the field 20, camera 2 is placed behind the goal on the right half of the field 20R, and camera 3 is placed behind the goal on the left half of the field 20L.

[0096] Fig. 13 is a flowchart showing a framing operation in soccer. Fig. 14 is a diagram showing an example of a framing operation for each camera in a case where three cameras are used in soccer.

[0097] 13, the user sets a favorite player as an initial setting (step 401). The control unit 14 determines whether a favorite player has been set (step 402).

[0098] If a favorite player has not been set (NO in step 402), the control unit 14 executes framing operation 7 (step 403). As shown in Fig. 14, framing operation 7 causes camera 1 to frame the field 20R or L on which the ball is located, camera 2 to frame the area around the ball holder while taking a close-up shot, and camera 3 to frame the area around the ball holder while taking a close-up shot.

[0099] The framing of the camera 2 in the framing operation 7 may be performed so that in addition to the ball holder, two or three other players around the ball holder are also framed.

[0100] If a favorite player has been set (YES in step 402), it is determined whether the ball is separated from the favorite player (step 404). For example, this is determined based on the photographic information to determine whether the ball is separated from the favorite player by more than one-fourth of the field 20.

[0101] If the ball is not separated from the favorite player (NO in step 404), it is determined whether the ball is on the right side (field 20R) (step 405). The fact that the ball is on either field 20R or field 20L indicates which goal the ball is approaching, i.e., whether the team is attacking or being attacked.

[0102] If the ball is on the right side (YES in step 405), the control unit 14 executes framing operation 8 (step 406). Framing operation 8 causes camera 1 to frame the field 20R or L where the ball is located, camera 2 to frame the favorite player at the center, and camera 3 to frame the area around the ball holder while taking a close-up. The reason for performing this framing control for multiple cameras is that camera 1, located in the center of the field, is suited to capturing a panoramic view of the field, so camera 1 is tasked with framing the entire field 20R or L. In a situation where the favorite player is likely to be present on the right side in addition to the ball, camera 2, which is closer to the favorite player than camera 3, is prioritized over framing the area around the ball holder with a camera located closer to the player. Therefore, camera 2, which is closer to the favorite player than camera 3, is configured to frame the favorite player, and the remaining camera 3 is configured to frame the area around the ball holder. If it is more important to have the favorite player framed by the nearest camera, the favorite player may be framed by camera 1 depending on the position of the favorite player.

[0103] When framing operation 8 from step 405 to step 406 is executed, camera 1 frames field 20R.

[0104] If the ball is on the left side (NO in step 405), the control unit 14 executes framing operation 9 (step 407). Framing operation 9 causes camera 1 to frame field 20R or L where the ball is located (field 20L in the case of step 407), camera 2 to frame while taking a close-up of the area around the ball holder, and camera 3 to frame with the favorite player at the center.

[0105] If the ball is far from the favorite player (YES in step 404) and if the ball is on the left side (NO in step 408), the control unit 14 executes framing operation 8 (step 406). When framing operation 8 is executed from step 408 to step 406, the camera 1 frames the field 20L.

[0106] If the ball is far from the favorite player (YES in step 404) and the ball is on the right side (YES in step 408), the control unit 14 executes framing operation 9 (step 409).

[0107] FIG. 15 is a schematic diagram showing the locations of the cameras when two cameras are used.

[0108] As shown in Figure 15, when two cameras are used, they are positioned so that they can frame the fields 20R and 20L. In Figure 15, the cameras are positioned around the corners (diagonally) of the field 20 in order to frame the players and the game situation from various angles.

[0109] Fig. 16 is a flowchart showing a framing operation for two cameras in soccer. Fig. 17 is a diagram showing an example of a framing operation for each camera in two cameras in soccer.

[0110] 16, the user sets a favorite player as an initial setting (step 501). The control unit 14 determines whether a favorite player has been set (step 502).

[0111] If a favorite player has not been set (NO in step 502) and the ball is not on the right side (NO in step 503), the control unit 14 executes the framing operation 10 (step 504). As shown in Fig. 17, the framing operation 10 causes the camera 1 to frame the area around the ball holder while zooming in, and causes the camera 2 to frame the field 20R or L where the ball is located.

[0112] If a favorite player has not been set (NO in step 502) and the ball is on the right side (YES in step 503), the control unit 14 executes the framing operation 11 (step 505). As shown in Fig. 17, the framing operation 11 causes the camera 1 to frame the field 20R or L where the ball is located (in the case of step 505, the field 20R is framed), and causes the camera 2 to frame the area around the ball holder while taking a close-up shot.

[0113] If a favorite player has been set (YES in step 502), it is determined whether the ball is away from the favorite player (step 506). If the ball is not away from the favorite player (NO in step 506), it is determined whether the ball is on the right side (field 20R) (step 507).

[0114] If the ball is on the right side (YES in step 507), the control unit 14 executes the framing operation 12 (step 508). The framing operation 12 causes the camera 1 to frame the image around the favorite player, and causes the camera 2 to frame the image so as to overlook the area around the ball holder.

[0115] If the ball is on the left side (NO in step 507), the control unit 14 executes the framing operation 13 (step 509). The framing operation 13 causes the camera 1 to frame the image so as to overlook the area around the ball holder, and causes the camera 2 to frame the image so as to center on the favorite player.

[0116] If the ball is far from the favorite player (YES in step 506) and the ball is on the left side (NO in step 510), the control unit 14 executes framing operation 12 (step 508). If the ball is far from the favorite player (YES in step 506) and the ball is on the right side (YES in step 510), the control unit 14 executes framing operation 13 (step 511).

[0117] In this way, in the second embodiment, the framing operation pattern is changed based on the position of the soccer ball, the position of the favorite player, and the change between offense and defense (whether the ball is on the left or right side of the field 20). This makes it possible to acquire images suitable for various situations.

[0118] Other Embodiments The present technology is not limited to the above-described embodiments, and various other embodiments can be realized.

[0119] In the above embodiment, sports such as baseball and soccer are used as examples of content. However, the present invention is not limited to these examples. The imaging system 100 may also be used for music-related content such as concerts and live performances. For example, feature information such as introductions of band members, guitar solos, the chorus of a song, and audience applause may be detected, and framing operations may be performed based on the feature information. Furthermore, for example, the framing operation may allow a user to set a favorite band member (corresponding to the favorite player in the first embodiment) as the initial setting. In this case, the framing operation may involve a camera 1 capturing all the band members, a camera 2 capturing the favorite band member, and a camera 3 capturing the audience.

[0120] In the above embodiment, the triggers for switching the framing operation were whether the batter was right-handed or left-handed, whether the batter had hit, and whether there were runners on base. However, the present invention is not limited to this, and various plays during a game, such as a manager's sign, a pinch runner, a pinch hitter, a defensive shift, or a double play, may also serve as triggers as characteristic information. Furthermore, a framing operation appropriate for each of these pieces of characteristic information may be set.

[0121] In the above embodiment, video editing such as cutting is performed by the information processing device 10. However, the present invention is not limited to this, and video editing may be performed by an external device. For example, by uploading baseball video obtained by framing operations to the cloud, the video may be edited by an external device having the functionality of the editing unit 15.

[0122] The configurations of the setting unit, detection unit, control unit, etc. described with reference to the drawings are merely one embodiment and can be modified as desired without departing from the spirit of the present technology. In other words, any other configurations, algorithms, etc. for implementing the present technology may be adopted.

[0123] It should be noted that the effects described in this disclosure are merely examples and are not limiting, and other effects may also be present. The description of multiple effects above does not necessarily mean that these effects are exhibited simultaneously. It means that at least one of the effects described above can be obtained depending on the conditions, etc., and of course, effects not described in this disclosure may also be exhibited.

[0124] It is also possible to combine at least two of the features of each embodiment described above. In other words, the various features described in each embodiment may be combined in any manner without distinguishing between the embodiments.

[0125] Note that the present technology may also have the following configurations. (1) An information processing device having: an acquisition unit that acquires shooting information acquired from one or more shooting units that shoot content; a detection unit that detects feature information indicating features of the content based on the shooting information; and a control unit that controls each of the shooting units based on the feature information. (2) The information processing device described in (1), wherein the feature information includes a target object related to the content, and the control unit controls the shooting units to perform a framing operation on the target object related to the content. (3) The information processing device described in (2), wherein the target object includes a character related to the content, and the control unit controls at least one of the imaging units to perform a framing operation on the character. (4) The information processing device described in (3), wherein the feature information includes at least one of a situation of the content or an action of the character. (5) The information processing device described in (4), wherein the framing operation includes a plurality of patterns, and the control unit switches the pattern of the framing operation based on the feature information. (6) The information processing device according to (2), wherein the framing operation includes zooming in and tracking the target object. (7) The information processing device according to (2), further comprising a setting unit that performs initial settings related to shooting of the content, which becomes the feature information, and the setting unit presents setting information on a user interface that allows the initial settings to be set. (8) The information processing device according to (7), wherein the initial settings include the number of the shooting units, a person to be shot of the content, and a time for framing the person to be shot. (9) The information processing device according to (8), wherein, when the person to be shot is set, the control unit controls at least one of the shooting units to perform a framing operation on the person to be shot.(10) The information processing device according to (8), wherein the initial setting includes a positional relationship of the shooting units, and the setting unit presents, on the user interface, a location for the shooting units suitable for the content. (11) The information processing device according to (9), wherein the initial setting includes an allocation of time for framing a person who is a subject of shooting in the content and another person different from the person who is a subject of shooting in the content, and the control unit controls to perform the framing operation based on the allocation of time. (12) The information processing device according to (1), wherein the content includes sports or music. (13) An information processing method executed by a computer system, comprising: acquiring shooting information acquired from one or more shooting units that shoot content; detecting feature information indicating features of the content based on the shooting information; and controlling each of the shooting units based on the feature information. (14) A program that causes a computer system to execute the steps of: acquiring shooting information obtained from one or more shooting units that shoot content; detecting feature information that indicates features of the content based on the shooting information; and controlling each of the shooting units based on the feature information.

[0126] DESCRIPTION OF SYMBOLS 1: Imaging unit 5: User device 10: Information processing device 11: Setting unit 13: Detection unit 14: Control unit 100: Imaging system

Claims

1. An information processing device having: an acquisition unit that acquires shooting information obtained from one or more shooting units that shoot content; a detection unit that detects feature information that indicates the features of the content based on the shooting information; and a control unit that controls each of the shooting units based on the feature information.

2. An information processing device according to claim 1, wherein the feature information includes a target object related to the content, and the control unit controls the imaging unit to perform a framing operation on the target object.

3. An information processing device according to claim 2, wherein the target object includes a character related to the content, and the control unit controls at least one of the imaging units to perform a framing operation on the character.

4. An information processing device according to claim 3, wherein the feature information includes at least one of the situation of the content or the action of the characters.

5. An information processing device according to claim 4, wherein the framing operation includes a plurality of patterns, and the control unit switches the pattern of the framing operation based on the characteristic information.

6. An information processing device according to claim 2, wherein the framing operation includes zooming and tracking the target object.

7. An information processing device according to claim 2, further comprising a setting unit for performing initial settings related to the shooting of the content, which constitutes the characteristic information, and the setting unit presents setting information on a user interface that allows the initial settings to be set.

8. An information processing device according to claim 7, wherein the initial settings include the number of the image capturing units, people to be captured in the content, and a time for framing the people to be captured.

9. An information processing device according to claim 8, wherein the control unit controls at least one of the photographing units to perform a framing operation on the person to be photographed when the person to be photographed is set.

10. An information processing device according to claim 8, wherein the initial setting includes a positional relationship of the image capturing unit, and the setting unit presents, on the user interface, a position of the image capturing unit suitable for the content.

11. An information processing device according to claim 9, wherein the initial setting includes an allocation of time for framing a person who is the subject of photography for the content and another person different from the person who is the subject of photography for the content, and the control unit controls the framing operation to be performed based on the allocation of time.

12. An information processing device according to claim 1, wherein the content includes sports or music.

13. An information processing method executed by a computer system, which comprises: acquiring shooting information obtained from one or more shooting units that shoot content; detecting characteristic information indicating the characteristics of said content based on said shooting information; and controlling each of said shooting units based on said characteristic information.

14. A program that causes a computer system to execute the steps of: acquiring shooting information obtained from one or more shooting units that shoot content; detecting feature information that indicates the features of the content based on the shooting information; and controlling each of the shooting units based on the feature information.

Images