Information processing device, information processing method, and program

Abstract

This information processing device comprises a calculation unit that performs: satisfaction determination processing for analyzing an image in which a performer is captured as a subject to determine whether a content distribution condition is satisfied; distribution destination determination processing for determining, when the satisfaction determination of the content distribution condition is made, a distribution destination of the content on the basis of an action of the performer from which the satisfaction determination was made; and content processing for selecting or generating the content to be distributed to the set distribution destination.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly to a service that provides content.

[0002] In recent years, services that distribute various types of content over networks such as the Internet have become widespread. For example, Patent Document 1 discloses a technology for improving a user's viewing experience in content distribution.

[0003] Japanese Patent Application Laid-Open No. 2021-192470

[0004] However, in the current situation where various entertainment content such as live music, plays, and movies is widely distributed, users want to obtain content that gives them a sense of exclusivity, rather than content that anyone can watch.

[0005] Therefore, this disclosure proposes a technology for providing a service that enables users who are audience members of a live music concert or the like to acquire content based on their experiences with the performer.

[0006] The information processing device according to the present technology includes a calculation unit that performs a fulfillment determination process of analyzing a captured image of a performer as a subject and determining whether a content distribution condition is fulfilled, a distribution destination determination process that determines a distribution destination of the content based on the behavior of the performer that caused the fulfillment determination when the content distribution condition is fulfilled, and a content process that selects or generates content to be distributed to the set distribution destinations. This enables a service in which, when the behavior of the performer fulfills the content distribution condition, the distribution destination of the content is determined according to the behavior.

[0007] 1 is an explanatory diagram of a content providing system according to an embodiment of the present technology. FIG. 1 is an explanatory diagram of a live venue when the content providing system according to an embodiment is applied. FIG. 2 is an explanatory diagram of area settings of a live venue to which the embodiment is applied. FIG. 3 is an explanatory diagram of an initial screen according to an embodiment. FIG. 4 is an explanatory diagram of a standby screen according to an embodiment. FIG. 5 is an explanatory diagram of a notification screen when content is distributed according to an embodiment. FIG. 6 is an explanatory diagram of a content screen according to an embodiment. FIG. 7 is an explanatory diagram of an album screen according to an embodiment. FIG. 8 is an explanatory diagram of a functional configuration of a server device according to an embodiment. FIG. 9 is an explanatory diagram of direction determination according to an embodiment. FIG. 10 is a block diagram of an information processing device according to an embodiment. FIG. 11 is a flowchart of an example of content providing processing of a server device according to an embodiment. FIG. 12 is a flowchart of an example of analysis processing according to an embodiment. FIG. 13 is a flowchart of an example of distribution destination determination processing according to an embodiment. FIG. 14 is a flowchart of an example of content processing according to an embodiment.

[0008] The following describes the embodiment in the following order: <1. System configuration> <2. Screen example of content providing service> <3. Server device configuration> <4. Processing example> <5. Summary and modifications>

[0009] In this disclosure, "image" refers to both video and still images. Furthermore, "image" refers to an image that is actually displayed on a screen, but "image" in the signal processing process and transmission path leading up to display on the screen refers to image data. In the services of the embodiments, content is mainly provided as images, but may also include still images, video, and pseudo-videos made up of a small number of still images. These images may or may not include audio.

[0010] Furthermore, a "performer" refers to a person who performs in front of an audience, and is not limited to a real person, but in the embodiments, a singer is used as an example. Furthermore, the "behavior" of a performer in this disclosure is a term that comprehensively describes any action, such as facial expressions, posture, a series of posture changes, vocalizations, etc.

[0011] <1. System Configuration> Fig. 1 shows an overview of a content providing system 1 according to an embodiment. In this embodiment, an example is a service that provides users with content based on their experiences at a live music concert. In this case, a singer or the like is assumed to be the performer, and an audience is assumed to be the users who enjoy the services provided by the content providing system 1. However, the content providing system 1 according to this embodiment can also be applied to performance venues other than live music concerts, and application to live music concerts is just one example.

[0012] 1 shows the following hardware elements that make up the content providing system 1: a server device 2, a database unit 3 (hereinafter, database will be referred to as "DB"), a local controller 4, a terminal device 5, an imaging device 6 (hereinafter, camera 6"), a beacon terminal 7, and a network 8. Note that these are only examples of the hardware elements that make up the content providing system 1, and some of the elements may not be included, or hardware elements other than those shown may be added.

[0013] In the content providing system 1 shown in Fig. 1, the server device 2, the local controller 4, and the terminal device 5 are capable of communicating with each other, for example, via a network 8. Note that these devices may communicate with each other via wired or wireless communication instead of network communication.

[0014] This content providing system 1 is a system that realizes a service (hereinafter referred to as a "performance-linked service") that provides content to users with terminal devices 5 based on the experiences of audience members caused by the actions of performers such as singers. For example, in this performance-linked service, content is distributed to corresponding users in response to actions such as "responses" received from performers such as singers at a live venue. "Responses" here is an abbreviation of "response," and refers to the individual actions of singers and other performers reacting to the cheers and other support of their fans during a live performance.

[0015] For example, when an idol singer sends a response to a certain audience member, the server device 2 selects or generates content related to that singer, identifies the user to whom the response was sent, and assigns the content to that user. This allows the user who received the response to obtain special content triggered by the response. In addition to enjoying the live performance itself, users can also enjoy obtaining content provided by the live performance linked service.

[0016] The cameras 6 in the content providing system 1 are configured as digital camera devices having imaging elements such as CCD (Charge Coupled Devices) sensors or CMOS (Complementary Metal-Oxide-Semiconductor) sensors, and obtain captured images as digital data. For example, each camera 6 obtains captured images as moving images.

[0017] This camera 6 captures images of the stage and audience floor from various positions, for example, at a live concert venue. Although multiple cameras 6 are shown, it is sufficient to have at least one camera 6. However, it is desirable to have multiple cameras 6, as this improves the accuracy of determining the behavior of performers and enables the generation of diverse content.

[0018] For example, in the content providing system 1, skeletal capture data of the subject performer is extracted from the image captured by such camera 6, and the performer's behavior, such as dance, pose, specific posture, etc., can be determined based on the skeletal capture data. In addition to the performer, the movement of the microphone, musical instrument, stage set, etc. used by the performer may also be determined.

[0019] In recent years, in the field of sports such as soccer and basketball, a technology known as EPTS (Electronic Performance and Tracking Systems) has become known that estimates the posture and position of players and referees, as well as the position and rotation of the ball, from a specified field based on images taken by specially installed cameras and information from sensors (acceleration sensors and GPS sensors) attached to people (players) and objects (balls) involved in the game.

[0020] Specifically, the camera 6 may capture images to obtain such EPTS data as skeleton capture data. This allows the performer's behavior to be determined precisely. The images captured by the camera 6 can also be used as live images of a live music concert, etc.

[0021] The local controller 4 is shown as a device that performs communication processing at the live venue. For example, images captured by each camera 6 are supplied to the local controller 4, and then transmitted to the server device 2 by the local controller 4. Note that each camera 6 may be equipped with a communication function or may be individually connected to a corresponding communication device, and the captured images may be transmitted to the server device 2. In this case, the local controller 4 may not be provided.

[0022] One or more beacon terminals 7 are placed at predetermined locations within the live venue and output beacon signals. These beacon terminals 7 are mainly used to determine the locations of individual users who are audience members. The beacon signal transmitted from the beacon terminal 7, i.e., a signal including a terminal ID (identification) unique to the beacon terminal 7, is received by a terminal device 5 carried by the user and transmitted from the terminal device 5 to the server device 2. This allows the server device 2 to determine the location of each terminal device 5.

[0023] The terminal device 5 is an information processing device such as a smartphone, tablet device, or personal computer, but as described above, it is assumed that the terminal device 5 is a device carried by a user who is an audience member. The display unit 5a of the terminal device 5 displays content images, etc., which will be described later, by the performance linkage service.

[0024] As a performance linkage service, the server device 2 performs various processes for distributing content to the terminal device 5. For example, as processes of a processor shown as a calculation unit 10, the server device 2 performs processes such as analyzing images and sounds captured by the camera 6 to determine the behavior of the performer, determining users to whom the content is to be distributed, selecting and generating content, and distributing content.

[0025] The server device 2 can be an information processing device that performs cloud computing, that is, a cloud server.

[0026] However, the processing for displaying content images and the like on the terminal device 5 may be performed by an information processing device other than the cloud server. For example, it is conceivable that an information processing device such as a personal computer installed at a live venue has the functionality of the server device 2. Furthermore, it is also conceivable that the terminal device 5 also has the functionality of the server device 2 by using an application program and performs processing up to content acquisition and the like.

[0027] The DB unit 3 collectively refers to storage that stores various DBs used by the server device 2 in the performance linkage service. For example, there are DBs for various image and audio content related to performers, a user DB that stores information on users who have registered for the performance linkage service, etc. The DB unit 3 may also be provided with a performer DB that stores information on performers, such as the date, time, location, theme, program, and other information about various performers' live performances.

[0028] 2 shows a live music venue 15. For example, the venue 15 is roughly divided into a stage 16 where performers such as singers stand, and a floor 17 where the audience is located. Cameras 6 are positioned so that they can capture images of the stage 16 and floor 17 from various positions.

[0029] In the live performance linked service, the server device 2 divides the floor 17 into a plurality of divided areas 18 as a specific example of determining the individual locations of users on the floor 17. For example, FIG. 3 shows a state in which areas AR11 to AR44 are set as the divided areas 18. The server device 2 then determines in which divided area 18 the user, i.e., each terminal device 5, is located, using the terminal ID of the beacon terminal 7 transmitted from each terminal device 5. Note that such division into areas and use of the beacon terminal 7 is just one example, and location information transmitted from each terminal device 5 may also be used. For example, the terminal device 5 may obtain location information using a GPS (Global Positioning System) signal and transmit it to the server device 2. Furthermore, if there are seats on the floor 17, sensors on the seats may also be used.

[0030] 2, the server device 2 identifies the users in that direction, for example, when a performer on the stage 16 in Fig. 2 points in a certain direction as indicated by the arrow DR. For example, the server device 2 determines the divided area 18 corresponding to the direction of the arrow DR, and sets the users in that divided area 18 as the recipients of the content.

[0031] 2. Screen Examples of Content Providing Service The following describes screen examples on the terminal device 5. In order to receive the performance-linked service provided by the content providing system 1, the user starts up an application program provided by the service on the terminal device 5, or accesses and logs in to a website provided by the service.

[0032] As a result, an initial screen 60 such as that shown in Fig. 4 is displayed on the terminal device 5. The initial screen 60 displays a service logo 65, an initial image 66, and the like. The initial image 66 may always be a specific image, but it may also be an image of a performer appearing at a live venue to which the service is applied. In other words, an image of the performer may be used as the initial image 66 for each event, such as a live performance.

[0033] For example, the server device 2 may determine the current location of the logged-in user, determine a live performance or the like that is about to take place or is currently taking place at that location, determine the performer, and set the initial image 66 based on that.

[0034] 5 shows, for example, a standby screen 61 during a demonstration. The standby screen 61 displays, for example, a location image 67 indicating the location of the user who owns the terminal device 5. That is, the server device 2 determines the location of the terminal device 5 as described above, generates a location image 67 indicating the location, and displays it on each terminal device 5.

[0035] 6 shows a notification screen 62 when content distribution has been performed. The notification screen 62 displays a mark 68 indicating content acquisition, notifying the user of the content distribution. At the same time, the terminal device 5 may be vibrated. For example, if the performer points in a certain direction and a user in that direction is designated as a content distribution recipient, the notification screen 62 shown in FIG. 6 will be displayed on the terminal device 5 of that user.

[0036] FIG. 7 shows a content screen 63. For example, the user can view the acquired content on the terminal device 5 by tapping the mark 68 on the notification screen 62 in FIG. 6 or by performing a predetermined operation thereafter. The figure shows an example in which a content image 69 is displayed. Possible content includes a still image without audio, a still image with audio, a video with audio, a video without audio, and an audio file. The content of the content image 69 may include various examples such as a live-action image of the performer, an animated image, a virtual image, text such as a signature, or a graphic. Note that the content itself or link information to the content may be transmitted to the terminal device 5 at the time of distribution.

[0037] 8 shows an album screen 64. This is a screen that displays a list of content owned by a user, for example, and displays one or more content images 69. The content files themselves may be stored in the terminal device 5 after actually being downloaded to the terminal device 5, or may be managed by the server device 2 in the DB unit 3, for example, linked to a user ID. In either case, content files acquired by each user are saved in the local storage of the user's terminal device 5 or in cloud storage provided by the DB unit 3. The album screen 64 displays a list of content acquired and stored for each user, allowing the user to check the content they own.

[0038] 3. Configuration of Server Device> The following describes functions that the server device 2 has to perform processing to provide the above-described screens to the terminal device 5, for example.

[0039] 9 shows functions realized by a program in the computing unit 10 of the server device 2. The computing unit 10 is provided with a live performance linked service function 40, which includes a user management function 41, an analysis function 42, a content setting function 43, and a distribution control function 44.

[0040] The user management function 41 is a function for managing user information and acquiring information for determining the user's position. For example, the user management function 41 performs processing for creating a database and managing user information, such as identification information of the terminal device 5, user attributes, and content owned by the user, corresponding to the user ID. The user management function 41 also performs processing for acquiring information for determining the position, such as beacon signals and GPS data, sequentially transmitted from the terminal device 5, and estimating the current position of the user.

[0041] The analysis function 42 analyzes the images and accompanying audio captured by the camera 6 and determines whether the content distribution conditions are met. In particular, the analysis function 42 analyzes the images and audio of the performer to detect various behaviors of the performer, such as the performer's movements, posture, facial expressions, line of sight, voice, actions, etc. If a specific behavior set as a content distribution condition is detected, it is determined that the content distribution conditions are met.

[0042] For example, the analysis function 42 detects the following actions as content distribution conditions: A pose or action by a performer pointing in a certain direction A performer gazing or smiling in a certain direction for a predetermined period of time or more (for example, 2 to 3 seconds or more) An action by a performer waving or winking in a certain direction An action by a performer calling out in a certain direction An action by a performer requesting a call and response An action by a performer pointing a microphone or the like in a certain direction An action by a performer stirring up excitement in the audience

[0043] The above are just some examples, but it is conceivable that content distribution conditions could include actions by the performer that are presumed to be responses to the audience, actions taken to liven up the live performance, etc. Content distribution conditions may be set, for example, according to the type of performance.

[0044] Such an analysis function 42 can determine the performer's actions by analyzing live images captured by the camera 6. For example, the calculation unit 10 can use techniques such as 3D human pose estimation, which extracts human joints and facial features as key points from images and estimates posture by connecting the obtained key points. For example, Figure 10 shows examples of posture estimation based on key point extraction. Depending on each posture, the direction the performer is pointing can be determined, as indicated by the arrow. The calculation unit 10 can also generate the above-mentioned EPTS data and determine the performer's pose and action from the skeleton estimation data.

[0045] Furthermore, the calculation unit 10 can determine whether the performer is smiling, winking, etc. by analyzing the display of the face. In the case of facial expression analysis, the direction in which the performer is pointing can be determined by analyzing the direction of the performer's head and line of sight.

[0046] Depending on the analysis function 42, individual performers can also be identified. For example, if the performers are a group of singers, it can also identify which performer has taken action such as responding. Furthermore, depending on the analysis function 42, the performer's position, i.e., their position on the stage 16, can also be determined in real time. For example, it is possible to determine the position of each individual performer by using an image of the stage 16 and identifying each individual performer.

[0047] The analysis function 42 may analyze the vocal quality of the performer, specific words spoken, or cheers from the audience, and determine the content distribution conditions.

[0048] The content setting function 43 is a function for preparing content to be distributed when content distribution conditions are satisfied. Specifically, when the content distribution conditions for a certain performer are satisfied, the content setting function 43 performs a process of selecting content to be distributed from among the content related to that performer, or a process of generating content related to that performer.

[0049] The distribution control function 44 performs processing to determine the distribution destination of the prepared content depending on whether the content distribution conditions are satisfied. That is, when it is determined that the content distribution conditions are satisfied, the distribution control function 44 determines the distribution destination of the content based on the behavior of the performer that caused the determination of satisfaction. For example, the distribution destination may be a user who is in a divided area 18 corresponding to the direction specified by the behavior of the performer.

[0050] The distribution control function 44 also performs processing to enable users who have been determined as distribution destinations to acquire the content, such as actually transmitting content files and link information to the content files to the terminal devices 5, transmitting information for downloading the content, and storing content files in the DB unit 3 as cloud storage linked to the users.

[0051] The computing unit 10 of the server device 2 also shows a shared service function 50 in addition to the performance-linked service function 40 described above. This shared service function 50 may include a function as an NFT (Non-Fungible Token) market. The NFT market function is a service that allows users to list and sell NFT works they have acquired or purchase NFT works that are being listed.

[0052] For example, in this embodiment, content generated by the performance-linked service function 40 can be exchanged with or purchased as NFT content by other people in the sharing service function 50. For this reason, the sharing service function 50 is implemented with functions such as a sharing service, an exchange service, payment, and buying and selling processing.

[0053] This function allows users to share, exchange, or buy and sell content acquired through the content providing system 1 with other users.

[0054] 9 is an example. Some of the functions of the calculation unit 10 shown in FIG. 9 can be processed on the terminal device 5 side. Also, the shared service function 50 does not have to be provided.

[0055] The following describes the configuration of an information processing device 70 used in the content providing system 1. Fig. 11 shows an information processing device 70 that functions as, for example, the server device 2 or the terminal device 5. The information processing device 70 can be configured as, for example, a dedicated workstation, a general-purpose personal computer, a mobile terminal device, or the like.

[0056] 11 executes various processes in accordance with programs stored in a ROM 72 or a nonvolatile memory unit 74 such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), or programs loaded from a storage unit 79 to a RAM 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes as appropriate.

[0057] The image processing unit 85 is configured as a processor that performs various types of image processing, such as image generation processing for video clips, image analysis processing for captured images, generation processing for animation images and CG images, DB (Data Base) processing, and image effect processing.

[0058] The image processing unit 85 can be realized by, for example, a CPU separate from the CPU 71, a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), an artificial intelligence (AI) processor, etc. The image processing unit 85 may be provided as a function within the CPU 71.

[0059] The CPU 71, ROM 72, RAM 73, nonvolatile memory unit 74, and image processing unit 85 are interconnected via a bus 83. To this bus 83, an input / output interface 75 is also connected.

[0060] An input unit 76 consisting of operators and operation devices is connected to the input / output interface 75. For example, the input unit 76 may be various operators and operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, a remote controller, etc. The input unit 76 detects user operations, and the CPU 71 interprets signals corresponding to the input operations.

[0061] The input / output interface 75 is also connected, either integrally or separately, to a display unit 77 such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) panel, and an audio output unit 78 such as a speaker.

[0062] The display unit 77 performs various displays as a user interface. The display unit 77 is configured, for example, by a display device provided in the housing of the information processing device 70 or a separate display device connected to the information processing device 70. The display unit 77 displays various images on the display screen based on instructions from the CPU 71. The display unit 77 also displays various operation menus, icons, messages, etc., i.e., a GUI (Graphical User Interface), based on instructions from the CPU 71.

[0063] A storage unit 79 configured with an SSD (Solid State Drive) or an HDD (Hard Disk Drive) or a communication unit 80 configured with a modem or the like may be connected to the input / output interface 75. The communication unit 80 performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, or the like.

[0064] A drive 82 is also connected to the input / output interface 75 as needed, and a removable recording medium 81 such as a flash memory, a memory card, a magnetic disk, an optical disk, or a magneto-optical disk is appropriately attached. The drive 82 allows data files such as image files and various computer programs to be read from the removable recording medium 81. The read data files are stored in the storage unit 79, and images and sounds contained in the data files are output on the display unit 77 and the audio output unit 78. In addition, the computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as needed.

[0065] In this information processing device 70, software can be installed via network communication by the communication unit 80 or via a removable recording medium 81. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, or the like.

[0066] For example, when the information processing device 70 is considered as the server device 2, the functions shown in FIG. 9 are provided by software, and the CPU 71 and image processing unit 85 execute processing as the calculation unit 10 using these functions.

[0067] 4. Processing Example An example of processing by the calculation unit 10 of the server device 2 will be described below. Fig. 12 shows an example of processing executed by the calculation unit 10 during, for example, a live performance by a performer. This is processing performed by the functions of the calculation unit 10 shown in Fig. 9.

[0068] During the live performance, images of the live venue 15 captured by the camera 6 are continuously transmitted to the server device 2. The calculation unit 10 performs analysis processing on the captured images and audio accompanying the captured images in real time, and repeatedly executes the processing of Fig. 12. Note that the screen on the terminal device 5 displays the standby screen 61 of Fig. 5 during the live performance.

[0069] In step S101, the calculation unit 10 analyzes the captured image and audio and performs a fulfillment determination process based on the analysis. That is, the calculation unit 10 analyzes the image and audio of the performer, and determines whether the performer's behavior fulfills the content distribution conditions.

[0070] The calculation unit 10 branches the process in step S102. If the content distribution conditions are not satisfied, the process returns to step S101 via step S106. Note that in step S106, the calculation unit 10 determines whether the process is finished, and ends the process in Fig. 12 when a performance such as a live music concert has ended. The calculation unit 10 returns to step S101 until it determines that the process is finished.

[0071] If it is determined that the content distribution conditions are satisfied as a result of the analysis process in step S101, the calculation unit 10 proceeds from step S102 to step S103 to perform a distribution destination determination process, which determines users to whom the content will be distributed in accordance with whether the content distribution conditions are satisfied this time.

[0072] In step S104, the calculation unit 10 performs content processing, which is processing to prepare content to be distributed to the determined distribution destinations, and specifically, processing to select content already stored in the DB unit 3 or the like, or processing to generate new content.

[0073] In step S105, the calculation unit 10 performs a process of distributing the selected or generated content to the users who have been determined as recipients. Specifically, one of the following processes or a combination thereof may be considered.

[0074] - The content file itself is sent to the terminal device 5. - Link information for the content file is sent to the terminal device 5, allowing the user to view or download the content file. - The content is managed in the DB unit 3 or the like by linking it to a user ID, so that the recipient user can access and view it. - The content file is stored in the recipient user's personal cloud storage.

[0075] The above is an example of the process of step S105, but the process is not limited to this. In any case, in step S105, the calculation unit 10 executes a process to make the user who is the distribution destination recognize that he or she has acquired the content.

[0076] In addition to the above processing, in step S105, the calculation unit 10 notifies the terminal device 5 of the recipient user of the content distribution. As a result, the notification screen 62 in Fig. 6 is displayed on the terminal device 5 at the timing of this content distribution processing. The user can view the content at any time by operating the terminal device 5. Note that in step S105, only the notification of content distribution may be sent to the terminal device 5, and the actual content distribution may be performed at a later time.

[0077] Furthermore, the distributed content may be, for example, NFT content, which users who acquire it may be able to exchange, buy, sell, etc. on the NFT market.

[0078] Steps S101, S103, and S104 will be described in detail with reference to Fig. 12. First, an example of the analysis process in step S101 is shown in Fig. 13.

[0079] In step S121, the calculation unit 10 performs an individual identification of the performer using the image captured by the camera 6. If there is only one performer on the stage 16, the individual identification of that performer is performed; if there are multiple performers, the individual identification of each performer is performed. This is done so that content can be distributed according to each performer. Therefore, it is advisable to identify the individual performers to the extent that it is possible to identify who each performer is. However, for example, if the performers are a group and all content to be distributed is content related to the entire group, it is conceivable that individual identification may not be performed.

[0080] The calculation unit 10 can also determine the position of each performer on the stage 16 by tracking each performer on the captured image while identifying the individual performer.

[0081] After identifying the individual performers, the calculation unit 10 executes the processes from step S122 onwards for each recognized individual.

[0082] In step S122, the calculation unit 10 determines the performer's facial expression from the captured image. For example, it determines the performer's gaze, smile, blink, wink, etc. In step S123, the calculation unit 10 determines the performer's posture and behavior. For example, as shown in FIG. 10, it estimates posture based on keypoint extraction and determines behavior as a series of posture changes. Furthermore, if the performer assumes a posture that makes them point in a certain direction, the calculation unit 10 also determines the pointing direction.

[0083] In step S124, the calculation unit 10 makes a determination about the performer's voice, such as whether the performer has uttered a particular word, whether it is a particular lyric in a song, or whether it is a call-and-response situation.

[0084] Although the diagram shows an example in which all of the facial expression determination in step S122, the posture / behavior determination in step S123, and the voice determination in step S124 are performed, it is sufficient for the analysis process to perform at least one of these.

[0085] In step S125, the calculation unit 10 determines whether the content distribution conditions are satisfied. That is, if the calculation unit 10 detects a facial expression, posture / behavior, or voice that satisfies the set content distribution conditions as a result of the determination in steps S122, S123, and S124, it determines that the content distribution conditions are satisfied. If the calculation unit 10 does not detect a facial expression, posture, or voice that satisfies the set content distribution conditions, it determines that the content distribution conditions are not satisfied. Depending on this result, the processing branches in step S102 of FIG. 12.

[0086] It is also conceivable that the content distribution conditions may be determined by AND or OR conditions for specific facial expressions, specific postures and behaviors, and specific voices. In other words, the content distribution conditions may be set in any way. It is desirable to set the content distribution conditions flexibly depending on the type and character of the performer. For example, if the performer is an idol singer, the conditions may be postures and facial expressions that correspond to responses, while if the performer is a heavy metal band, the conditions may be specific actions or facial expressions that intimidate the audience.

[0087] An example of the recipient determination process in step S103 in Fig. 12 is shown in Fig. 14. In step S141, the calculation unit 10 determines a target area for selecting users to whom content is to be distributed. The calculation unit 10 sets divided areas 18 by dividing the floor 17 as described in Figs. 2 and 3. The calculation unit 10 determines the divided area 18 that corresponds to the performer's behavior. For example, it determines which of areas AR11 to AR44 the divided area 18 indicated by the performer's behavior is.

[0088] For example, the calculation unit 10 determines the divided area 18 indicated by the behavior of the performer determined to satisfy the conditions in step S101 as the target area. The calculation unit 10 can determine the target area within the floor 17 by determining the performer's position at that time and the direction indicated by the performer's posture and line of sight. Alternatively, the calculation unit 10 may make the determination based on the performer's vocalization. For example, if the performer utters, "Everyone in the second floor seats," the divided area 18 corresponding to the second floor seats is determined to be the target area. If the behavior of the performer who has satisfied the content distribution conditions is not particularly directional, the entire floor 17 may be determined to be the target area, or the target area may be set randomly to add a game-like element. In either case, the calculation unit 10 determines the target area based on the performer's behavior.

[0089] In step S142, the calculation unit 10 determines whether or not a user (audience) exists in the determined target area. If no user exists, it is assumed that there is no user to be a distribution destination, and the process proceeds to step S106 in Fig. 12 as indicated by "c1." In other words, the process is the same as when the content distribution conditions are not satisfied.

[0090] If a user is present in the target area, the calculation unit 10 proceeds to step S143 and determines the users in the target area. The calculation unit 10 sequentially determines the position of each spectator user using a beacon signal, etc. The calculation unit 10 determines the users in the target area when the content distribution conditions are satisfied.

[0091] The calculation unit 10 also determines a special user in step S144. This is a process of selecting a special user from among the users in the target area. For example, the calculation unit 10 determines a user in the target area who is in a position that most closely matches the direction indicated by the performer as the special user. Alternatively, the calculation unit 10 may randomly select a special user from among the users in the target area.

[0092] In step S145, the calculation unit 10 determines users who will be distribution destinations. Basically, the calculation unit 10 determines users who are in the target area determined in step S143 as distribution destinations. Furthermore, the calculation unit 10 determines users who are in the target area as distribution destinations, but the special users selected in step S143 may also be distribution destinations of special content. Furthermore, the calculation unit 10 may select only special users as distribution destinations.

[0093] As the process of Fig. 14, the distribution destination determination process of step S103 of Fig. 12 is performed. Note that there may be an example in which the special user determination of step S144 is not performed.

[0094] Next, an example of the content processing in step S104 in Fig. 12 will be described with reference to Fig. 15 to Fig. 20. Content processing can be broadly divided into two types: selection processing of existing content and generation processing of new content. Fig. 15 and Fig. 16 show examples of selection processing, and Fig. 17 to Fig. 20 show examples of generation processing.

[0095] In the example of Figure 15, in step S161, the calculation unit 10 performs a process of selecting content to be distributed from existing content in accordance with the performer. The content may be video, still images, audio data, etc. related to the performer, but may also include content created in advance for a live performance or content that is always available. Prior to the live performance, the server device 2 obtains this content from the performer's office or the like and stores it in the DB unit 3 or the like. Then, when the content distribution conditions are met, content is selected from the stored content. For example, content may be selected randomly from existing content.

[0096] In the case where there are multiple performers (members), such as a singing group, content may be prepared for each member. If the behavior of a certain member satisfies the content distribution condition, the calculation unit 10 may select content to distribute from among the contents related to the member whose behavior satisfies the content distribution condition.

[0097] 16, the calculation unit 10 extracts content to be distributed from existing content according to the performer in step S162, and then selects content according to the live performance situation and various attributes in step S163. The following examples are possible for the calculation unit 10's process of selecting content to be distributed in step S163.

[0098] First, an example can be considered in which content is selected according to the date and time of the live performance, the day of the week, the location, the weather, the theme of the live performance tour, etc. For example, the calculation unit 10 prepares different content for each day of the week and selects content according to the day of the week.

[0099] Another example is to select content according to the behavior of the performer. For example, the calculation unit 10 prepares content corresponding to each type of behavior of the performer, such as pointing, smiling, winking, looking, and waving, and selects content according to the type of actual behavior. It is also possible to determine several specific poses of the performer and select content according to the pose.

[0100] In addition, it is also possible to select content according to the user to whom the content is distributed. For example, the calculation unit 10 prepares content according to the user's gender, age group, goods owned, color of clothing, location of the user, etc., and selects content according to the corresponding type.

[0101] Another example is to select content according to the content of the performance. For example, the calculation unit 10 prepares content according to the type of song or talk when the performer takes an action that satisfies the content distribution condition, and selects content according to the relevant content of the performance.

[0102] Next, an example of generating content will be described. In the example of Fig. 17, the calculation unit 10 performs a process of selecting a scene corresponding to the performer's actions from a live image, i.e., an image captured by the camera 6, in step S171, and then extracts a video or still image to generate the content in step S172.

[0103] For example, in step S171, the calculation unit 10 determines a scene in the captured image that includes a performer's behavior that satisfies the content distribution conditions. Then, in step S172, the calculation unit 10 cuts out the scene so that it becomes a video clip of an appropriate length, and creates video content. Alternatively, the calculation unit 10 cuts out a frame at the moment of the performer's behavior from the scene, and creates still image content.

[0104] This allows images from a live performance to be distributed as content. When generating content from captured images in this way, AI processing may be performed to determine whether the selected sections or frames are appropriate. For example, when creating still images, it is conceivable to exclude frames showing the moment when the eyes are closed from the selection, or not select poses designated by the performer or their agency. Conversely, it is also possible to use AI processing to determine frames showing the moment when a performer smiles or a so-called "finishing pose."

[0105] 18 shows an example of further editing using captured images. In step S171, the calculation unit 10 determines a scene in the captured image that includes a performer's behavior that satisfies the content distribution conditions. In step S174, the calculation unit 10 cuts out the scene so as to create a video clip of an appropriate length. In step S175, the calculation unit 10 performs editing on the cut video clip or frame to create video content or still image content.

[0106] Examples of editing processes include adding additional images, such as a performer's signature, to videos or still images. The content of the additional images could be based on the date, time, day of the week, location, weather, or theme of the live tour.

[0107] Furthermore, the content of the additional images may be images corresponding to the performer's actions. For example, the calculation unit 10 generates additional images corresponding to the performer's actions, such as pointing, smiling, winking, looking, and waving, and adds them through editing processing according to the type of actual action. Additional images may also be generated according to the performer's pose.

[0108] Another example is possible in which an additional image is generated or selected depending on the user to whom the content is distributed. For example, the calculation unit 10 generates additional images depending on the user's gender, age group, items owned, color of clothing, location of the user, etc., and adds them through editing processing. Another example is possible in which an additional image is generated or selected depending on the content of a performance. For example, the calculation unit 10 generates or selects an additional image depending on the song played when the performer takes an action that satisfies the content distribution conditions.

[0109] The above is an example of generating an additional image, but editing processes for adding audio to video or still image content are also possible. The additional audio may be, for example, a voice message from a performer, but like the additional images described above, it may also be generated or selected according to various conditions.

[0110] Editing processing may also involve applying effects to the captured images, and the type of effect may be selected according to attributes related to the live performance, the type of behavior of the performer, information about the recipient users, the content of the performance, and the like.

[0111] By performing such editing processing, the user can obtain live images that have been specially edited as content.

[0112] 19 shows an example of generating a virtual image from a captured image to create a moving or still image content. In step S181, the calculation unit 10 generates a 3D model of the performer based on images captured by multiple cameras 6. Then, in step S182, the calculation unit 10 sets a viewpoint position for the 3D model. For example, the position of the user who is the recipient is set as the viewpoint position. In step S183, the calculation unit 10 generates a virtual image as an image seen from the viewpoint position set for the 3D model, and uses this as image content.

[0113] In recent years, a technique for free viewpoint video has become known in which a 3D model of a subject is generated from images captured from multiple viewpoints by multiple imaging devices, and an image is output in which the viewpoints of the 3D model are arbitrarily displaced. For example, when a performer is being imaged by multiple cameras 6, the free viewpoint video technique can be applied to the computing unit 10. Moreover, in this embodiment, the location of the user (divided area 18) is specified as the recipient user.

[0114] Therefore, by generating a 3D model of the performer and applying free viewpoint video technology, and by using the user's position as the viewpoint, it is possible to generate a virtual image that is close to what the user actually saw. By distributing this to users in the target area, the content will visualize the user's experience, and it is expected that user satisfaction will be high. For example, a user will be able to save the experience of receiving a response from an idol singer as it is.

[0115] It is also possible to create a 3D model in advance for a specific pose of the performer, set the viewpoint position based on the relationship between the performer's position and the user's position during the actual pose, and generate a virtual image.

[0116] 20 also shows an example of generating a virtual image from captured images to create moving or still image content. In step S191, the calculation unit 10 acquires images from multiple cameras 6, particularly captured images of the floor 17 in this case, and generates a 3D model of the floor scene. In this case, it is preferable to generate a 3D model that includes at least the audience in the target area to which the content is distributed. Then, in step S192, the calculation unit 10 sets a viewpoint position for the 3D model. For example, the position of the performer when performing an action that satisfies the content distribution conditions is set as the viewpoint position. In step S193, the calculation unit 10 generates a virtual image as an image seen from the viewpoint position set for the 3D model, and uses this as image content.

[0117] In other words, this example presents the user with an image of themselves as seen by the performer. This is special content that allows the recipient user to see how they looked to the performer, such as a singer, at the moment of response.

[0118] 17 to 20, examples have been given in which image content to be distributed is generated based on images captured by the camera 6, but it is also possible to generate image content without using captured images.

[0119] 14, the calculation unit 10 determines whether a user is a special user. A special user is a user who receives special content. For example, all users in a target area can obtain the content, but if a user is selected as a special user, he or she can obtain content that is different from that of other users.

[0120] The difference between normal content and special content may be created by the difference in the content processing of Figures 15 to 20. For example, normal content may be an image of the moment of a reply generated by the processing of Figure 17, while special content may be an image of the moment of a signed reply generated by the processing of Figure 18. Alternatively, both may be generated by the processing of Figure 18, but the content of the editing processing may be different.

[0121] By changing the content to be distributed in this way, it is possible to add a gambling element to the game, such as allowing players to acquire valuable content if they are lucky.

[0122] 5. Summary and Modifications The above embodiment provides the following effects.

[0123] The information processing device serving as the server device 2 of the embodiment includes a calculation unit 10 that performs a fulfillment determination process of analyzing captured images of a performer to determine whether the content distribution conditions are fulfilled, a distribution destination determination process of determining a distribution destination of the content based on the performer's behavior that caused the fulfillment determination when the content distribution conditions are fulfilled, and a content process of selecting or generating content to be distributed to the set distribution destinations (see FIG. 12). This enables a service to be realized in which content is provided to users who are destinations determined based on the performer's behavior, triggered by the performer's behavior. Particularly in the entertainment field, audiences and the like can enjoy services that allow them to obtain content that adds new value to the performer or content based on their own experiences.

[0124] In the embodiment, an example has been given in which the calculation unit 10 performs a distribution process in which the content selected or generated in the content processing is made available to the user selected as the distribution destination in the distribution destination determination process (see step S105 in FIG. 12). For example, the calculation unit 10 performs a process in which the content is considered to be owned by the user, such as sending the content directly to the terminal device 5 of the user selected as the distribution destination, making the content downloadable, sending link information for the content, or linking the content to a user ID and managing it in the DB unit 3. This allows the user to actually feel satisfied by owning the content generated by the performer's actions.

[0125] In the embodiment, an example was given in which the calculation unit 10 determines that the content distribution condition is satisfied when it is estimated from the captured image that the performer's body state is pointing at a portion of the audience (see FIG. 13 ). For example, when the performer assumes a posture, behavior, or gesture of pointing at the audience, or when he or she takes an action to liven up a portion of the audience, it can be evaluated that the performer's body state is pointing at a portion of the audience. In such cases, the performer can be considered to be in a state where a user can select a distribution destination, and by determining that the content generation condition is satisfied, it becomes possible to provide a service such as content distribution according to the performer's response, for example.

[0126] In the embodiment, an example was given in which the calculation unit 10 determines that the content generation condition is satisfied when it is estimated from the captured image that the performer's face is pointing at a portion of the audience (see FIG. 13 ). For example, if the performer's facial expression includes gazing, smiling, winking, etc. in the direction of a specific audience member, it can be determined that the performer's face is pointing at a portion of the audience member. In such a case, it can be said that the user who will be the distribution destination can be selected. Therefore, by determining that the content distribution condition is satisfied, it becomes possible to provide a service such as content distribution according to the performer's response.

[0127] In the embodiment, the calculation unit 10 also analyzes the audio accompanying the captured image and uses the analysis results to determine whether the content distribution conditions are met (see FIG. 13). For example, during a live performance, the audio emitted by a performer may indicate a special moment of excitement or may be directed at a specific audience member. By determining that such a case satisfies the content generation conditions, a service can be provided in which content is distributed in accordance with special moments, such as a live performance, or special moments for the audience.

[0128] In the embodiment, an example was given in which the calculation unit 10 determines the direction from the performer's behavior and determines the audience members in the determined direction as the distribution destinations (see FIG. 14). For example, from the performer's behavior such as pointing, waving, looking, or winking, which are presumed to be responses to the audience, the direction from the performer to the audience members in a specific direction can be estimated. Therefore, the audience members in that direction can be determined as the distribution destinations. This makes it possible to provide a service in which, for example, when a performer takes an action that is presumed to be a response to the audience members, content is distributed to the audience members who are the target of the response.

[0129] In the embodiment, the calculation unit 10 divides the area where the audience is located into multiple areas, determines the area that corresponds to the direction determined from the performer's behavior, and determines the audience members in that area as recipients of the distribution (see Figure 14). For example, it is difficult to identify the audience members who will be the target of the performer's response in a crowded audience, but by managing the audience positions by area, it becomes easier to determine the audience members to whom the distribution will be made.

[0130] In the embodiment, the calculation unit 10 determines as recipients spectators who are in an area corresponding to the direction determined from the performer's actions, and selects the spectator whose direction most closely matches the direction determined from the performer's actions and determines him or her as a special recipient (see Figure 14). When the positions of spectators are managed by dividing the venue into areas, it is expected that multiple spectators will become recipients, but by selecting a special recipient among them, it is possible to distribute content that gives the spectator a special sense of good fortune.

[0131] In the embodiment, an example was given in which the calculation unit 10 performs a process of selecting content to be distributed from a plurality of contents stored in association with a performer (see FIG. 15). The server device 2 prepares still images, videos, message audio files, etc. related to the performer in advance, and selects the content of the performer at the moment of the performer's response, etc., thereby achieving content distribution with a relatively simple process.

[0132] In the embodiment, an example was given in which the calculation unit 10 performs a process of selecting content from among a plurality of pieces of content stored in association with a performer, based on the performer's behavior that caused the determination that the content distribution conditions were met (see FIG. 16). By the calculation unit 10 selecting content from among the performer's content based on the performer's behavior, such as responses, the audience can obtain content according to the type of behavior, such as responses, and can experience the special feeling of preserving that moment.

[0133] In the embodiment, an example was given in which the calculation unit 10 performs processing to generate content from captured images of a performer (see FIG. 17). The calculation unit 10 generates content by converting a frame of the captured image of the performer into a still image file or a section of the captured image into a video clip. This allows the audience to obtain content such as an image capturing the moment of an action such as a response, and gives them the special feeling of preserving that moment.

[0134] In the embodiment, an example was given in which the calculation unit 10 performs an editing process on images extracted from captured images of a performer to generate content (see FIG. 18 ). The calculation unit 10 extracts certain frames from the captured images of the performer as still images or extracts a portion of the captured images as a video clip, and then edits these still images or videos. This allows for the generation of a variety of content using the captured images. For example, content tailored to the actual venue, such as a live performance, can be generated by adding images, text, or audio according to the date, time, season, location, theme, song played at the moment of response, and the attributes of the users to whom the content is distributed.

[0135] In the embodiment, an example was given in which the calculation unit 10 performs editing processing on an image extracted from a captured image of a performer based on the performer's behavior that caused the content distribution condition to be determined to be satisfied (see FIG. 18). By editing a still image or video from the captured image by adding images, text, audio, etc. that correspond to the performer's behavior, such as responses, the audience can obtain content that more emphasizes the moment of the performer's behavior, such as responses, and can feel special.

[0136] In the embodiment, the calculation unit 10 generates an image of the performer based on a captured image of the performer, with the viewpoint set to the audience's location (see FIG. 19). For example, the calculation unit 10 generates an image of the performer with a different viewpoint as a virtual image using a 3D model. This allows a user who acquires the content to obtain extremely special content, such as the image they were looking at at the moment of a response.

[0137] In the embodiment, the calculation unit 10 performs processing to generate an image of the audience at the distribution destination, with the performer's position as the viewpoint, based on the captured image (see FIG. 20 ). For example, the calculation unit 10 generates an image of the venue as a virtual image using a captured image or a 3D model, with the viewpoint set to the performer's position and looking in the direction of the audience at the distribution destination. This allows the user obtaining the content to obtain extremely special content, such as a representation of themselves as seen from the performer's perspective at the moment of response, etc.

[0138] In the embodiment, the server device 2 is a cloud server, but it may be configured as a local server provided at the live venue 15 or the like.

[0139] The technology of the content providing system 1 of the present disclosure is not limited to live music performances, but can also be applied to a variety of performances, such as entertainment and sports. For example, performers include singers, actors, entertainers, dancers, musicians, athletes, speakers, lecturers, magicians, and others who perform some form of entertainment in front of an audience, and the present technology is suitable for performance opportunities for these performers. In the case of athletes, it is conceivable to distribute content to spectators in the direction indicated by their actions, not based on their actions during play, but based on, for example, their responses to the cheers of the audience after a game.

[0140] Furthermore, performers do not have to be real people. They can be animals, or virtual people or characters. For example, this technology can be applied to the relationship between a singer and an audience in a metaverse space. In that case, this technology can be applied as a system in which a user can receive content distribution when they receive a response from a virtual singer to their virtual self at a live venue in the metaverse space.

[0141] 12 to 20 is executed by, for example, a CPU, a DSP (digital signal processor), an AI processor, or an information processing device 70 including these. That is, the program of the embodiment is a program that causes an information processing device to execute a fulfillment determination process that analyzes a captured image of a performer as a subject and determines whether a content distribution condition is fulfilled, a distribution destination determination process that, when the content distribution condition is fulfilled, determines a distribution destination of the content based on the behavior of the performer that caused the fulfillment determination, and a content process that selects or generates content to be distributed to the set distribution destinations.

[0142] With such a program, the information processing device 70 serving as the server device 2 of the embodiment can be realized in, for example, a computer device, a mobile terminal device, or other device capable of executing information processing.

[0143] Such a program can be pre-recorded on a hard disk drive (HDD) as a recording medium built into a computer or other device, or on a ROM within a microcomputer having a CPU. Alternatively, the program can be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card. Such removable recording media can be provided as so-called packaged software. Furthermore, such a program can be installed on a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0144] Furthermore, such a program is suitable for providing a wide range of information processing devices 70 constituting the server device 2 of the embodiment. For example, by downloading the program to a mobile terminal device such as a smartphone or tablet, an imaging device, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), or the like, these devices can function as the information processing device 70 constituting the server device 2 of the present disclosure.

[0145] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0146] The present technology can also be configured as follows: (1) An information processing device comprising a calculation unit that performs a fulfillment determination process of analyzing captured images of a performer as a subject and determining whether a content distribution condition is fulfilled, a distribution destination determination process that, when the content distribution condition is fulfilled, determines a distribution destination of the content based on the behavior of the performer that caused the fulfillment determination, and a content process that selects or generates content to be distributed to the set distribution destinations. (2) The information processing device described in (1) above, wherein the calculation unit performs a distribution process that makes the content selected or generated in the content process available to the users who are determined to be distribution destinations in the distribution destination determination process. (3) The information processing device described in (1) or (2) above, wherein the calculation unit determines, in the fulfillment determination process, that the content distribution condition is fulfilled when it is estimated from the captured images that the performer's body state is in a state where it is pointing at some of the audience. (4) The information processing device according to any one of (1) to (3), wherein the calculation unit determines that the content generation conditions are satisfied when it is estimated from the captured image that the performer's face is pointing at some of the audience members in the fulfillment determination process. (5) The information processing device according to any one of (1) to (4), wherein the calculation unit also analyzes audio accompanying the captured image in the fulfillment determination process and uses the analysis results to determine whether the content distribution conditions are satisfied. (6) The information processing device according to any one of (1) to (5), wherein the calculation unit determines a direction from the performer's behavior and determines audience members in the determined direction as distribution destinations in the distribution destination determination process. (7) The information processing device according to (6), wherein the calculation unit divides an area where the audience members are present into a plurality of areas, determines an area corresponding to the direction determined from the performer's behavior, and determines audience members in the determined area as distribution destinations. (8) The information processing device described in (7) above, wherein the calculation unit determines spectators in an area corresponding to the direction determined from the performer's actions as distribution recipients, and selects spectators whose direction most closely matches the direction determined from the performer's actions and determines them as special distribution recipients.(9) The information processing device according to any one of (1) to (8), wherein the calculation unit performs a process of selecting content to be distributed from a plurality of pieces of content stored in association with a performer in the content processing. (10) The information processing device according to any one of (1) to (8), wherein the calculation unit performs a process of selecting content from a plurality of pieces of content stored in association with a performer in the content processing, based on the behavior of the performer that caused a determination that a content distribution condition was satisfied. (11) The information processing device according to any one of (1) to (8), wherein the calculation unit performs a process of generating content from captured images of the performer in the content processing. (12) The information processing device according to any one of (1) to (8), wherein the calculation unit performs a process of generating content by performing an editing process on an image extracted from captured images of the performer in the content processing. (13) The information processing device according to (12), wherein the calculation unit performs an editing process on an image extracted from captured images of the performer in the content processing, based on the behavior of the performer that caused a determination that a content distribution condition was satisfied. (14) The information processing device according to any one of (1) to (8), wherein the calculation unit, in the content processing, generates an image of the performer based on a captured image of the performer, with the position of the audience at the distribution destination as a viewpoint position. (15) The information processing device according to any one of (1) to (8), wherein the calculation unit, in the content processing, generates an image of the audience at the distribution destination, with the position of the performer as a viewpoint position, based on the captured image. (16) An information processing method, wherein an information processing device performs: a fulfillment determination process that analyzes a captured image of the performer as a subject and determines whether a content distribution condition is fulfilled; a distribution destination determination process that, when the content distribution condition is fulfilled, determines a distribution destination of the content based on the behavior of the performer that caused the fulfillment determination; and a content process that selects or generates content to be distributed to the set distribution destinations.(17) A program that causes an information processing device to execute the following: a fulfillment determination process that analyzes captured images of a performer as the subject and determines whether content distribution conditions are fulfilled; a distribution destination determination process that, when the content distribution conditions are fulfilled, determines the distribution destination of the content based on the behavior of the performer that caused the fulfillment determination; and a content process that selects or generates content to be distributed to the set distribution destinations.

[0147] REFERENCE SIGNS LIST 1 Content providing system 2 Server device 3 DB unit 5 Terminal device 6 Camera 7 Beacon terminal 8 Network 10 Calculation unit 40 Demonstration linked service function 41 User management function 42 Analysis function 43 Content setting function 44 Distribution control function 60 Initial screen 61 Standby screen 62 Notification screen 63 Content screen 64 Album screen

Claims

1. An information processing device having a calculation unit that performs the following: a fulfillment determination process that analyzes captured images of a performer as the subject and determines whether the content distribution conditions are fulfilled; a distribution destination determination process that, when the content distribution conditions are fulfilled, determines the distribution destination of the content based on the behavior of the performer that caused the fulfillment determination; and a content process that selects or generates content to be distributed to the set distribution destinations.

2. The information processing device according to claim 1, wherein the calculation unit performs a distribution process that makes the content selected or generated in the content processing available to the user who is designated as the distribution destination in the distribution destination determination process.

3. The information processing device of claim 1, wherein the calculation unit determines that the content distribution conditions are satisfied when it is estimated from the captured image that the performer's physical state is pointing at some of the audience members in the satisfaction determination process.

4. The information processing device of claim 1, wherein the calculation unit determines that the content generation conditions are satisfied when it is estimated from the captured image that the performer's face is pointing at some of the audience members in the satisfaction determination process.

5. The information processing device according to claim 1, wherein the calculation unit also analyzes audio accompanying the captured image during the fulfillment determination process, and uses the analysis results to determine whether the content distribution conditions are fulfilled.

6. An information processing device according to claim 1, wherein the calculation unit, in the distribution destination determination process, determines a direction from the performer's actions and determines spectators in the determined direction as distribution destinations.

7. The information processing device of claim 6, wherein the calculation unit, in the distribution destination determination process, divides the area in which the audience is located into multiple areas, determines the area corresponding to the direction determined from the performer's behavior, and determines the audience in the corresponding area as the distribution destination.

8. An information processing device as described in claim 7, wherein the calculation unit determines spectators in an area corresponding to the direction determined from the performer's actions as distribution recipients, and selects spectators whose direction most closely matches the direction determined from the performer's actions and determines them as special distribution recipients.

9. An information processing device according to claim 1, wherein the calculation unit performs processing to select content to be distributed from a plurality of contents stored in association with a performer in the content processing.

10. The information processing device of claim 1, wherein the calculation unit performs a process of selecting content from among multiple pieces of content stored in association with a performer based on the behavior of the performer that caused a determination that the content distribution conditions have been met.

11. An information processing device according to claim 1, wherein the calculation unit performs processing to generate content from captured images of a performer in the content processing.

12. The information processing device according to claim 1, wherein the calculation unit performs the content processing by editing an image extracted from a captured image of a performer to generate content.

13. The information processing device according to claim 12, wherein the calculation unit, in the editing process, edits the image extracted from the captured image of the performer based on the behavior of the performer that caused the content distribution conditions to be determined to be satisfied.

14. The information processing device according to claim 1, wherein the calculation unit performs processing to generate an image of the performer based on a captured image of the performer, with the viewpoint being the audience position to which the content is distributed.

15. The information processing device according to claim 1, wherein the calculation unit performs processing to generate an image of the audience to whom the content is distributed, with the performer's position as the viewpoint, based on the captured image.

16. An information processing method in which an information processing device performs the following: a fulfillment determination process in which an image captured of a performer is analyzed to determine whether the content distribution conditions are fulfilled; a distribution destination determination process in which, when the content distribution conditions are fulfilled, the distribution destination is determined based on the behavior of the performer that caused the fulfillment determination; and a content processing process in which content to be distributed to the set distribution destinations is selected or generated.

17. A program that causes an information processing device to execute the following processes: a fulfillment determination process that analyzes captured images of a performer and determines whether content distribution conditions are fulfilled; a distribution destination determination process that, when the content distribution conditions are fulfilled, determines the distribution destination of the content based on the behavior of the performer that caused the fulfillment determination; and a content process that selects or generates content to be distributed to the set distribution destinations.

Images