Information processing system, program, and information processing method

Abstract

To provide an information processing system where participants warm up more in lives so called a live entertainment and a live event, etc., a program, and an information processing method.SOLUTION: An information processing system comprises a server 100, devices detecting data related to persons, and displays and cameras installed in a live hall. The server comprises: an operation data acquisition unit which acquires operation data on operations of a participant identified on the basis of body motion data on the body motion of participants detected by devices worn by the participants participating in a live; an intensity degree determination unit which determines an intensity degree indicative of an intensity degree of each of a plurality of participants in the live on the basis of a plurality of operation data of the plurality of participants acquired by the operation data acquisition unit; and an output data generation unit which generates output data based on the intensity degrees of the plurality of participants.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to an information processing system, a program, and an information processing method. [Background technology]

[0002] Patent Document 1 describes a technique for controlling a performance on stage by inputting comments from a user terminal viewing the stage via a network. [Prior art document] [Patent documents] [Patent Document 1] JP 2013-037670 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided an information processing system. The information processing system may include a motion data acquisition unit that acquires motion data related to the motions of participants participating in a live event, the motion data being detected by a device worn by the participant. The information processing system may include an intensity determination unit that determines an intensity level indicating a degree of intensity of the live event for each of the multiple participants, based on the multiple motion data of the multiple participants acquired by the motion data acquisition unit. The information processing system may include an output data generation unit that generates output data based on the intensity levels of the multiple participants.

[0004] The information processing system may include a learning execution unit that performs machine learning using body movement data related to the body movement of a person wearing the device when the person performs a movement and a type of the movement as learning data to generate a learning model in which body movement data is input and a type of movement is output. The information processing system may include a motion identification unit that identifies the movement of a participant participating in the live performance by inputting the body movement data detected by the device worn by the participant into the learning model. In the information processing system, the motion data acquisition unit may acquire the motion data related to the movement of the participant identified by the motion identification unit. In the information processing system, the motion identification unit may identify the type, amount, and magnitude of the movement of the participant, and the motion data acquisition unit may acquire the motion data including the type, amount, and magnitude of the movement of the participant identified by the motion identification unit.

[0005] In any of the information processing systems, the intensity determination unit may continuously determine the intensity level of each of the multiple participants during the live broadcast, and the output data generation unit may generate at least one of real-time data indicating the intensity levels continuously determined during the live broadcast in real time for at least some of the multiple participants, and aggregated data based on an aggregation result of the intensity levels of the multiple participants.

[0006] Any of the information processing systems may include a matching degree determination unit that determines a matching degree of the multiple participants in the live performance based on at least one of the motion data of each of the multiple participants acquired by the motion data acquisition unit and the intensity of each of the multiple participants determined by the intensity determination unit. In the information processing system, the output data generation unit may generate matching degree data based on the matching degree. In the information processing system, the matching degree determination unit may determine the matching degree based on at least one of the type of motion and timing of the motion of each of the multiple participants, and the intensity value and transition of the intensity of each of the multiple participants. The information processing system may include an imaging data acquisition unit that acquires imaging data of the participants captured while participating in the live performance. In the information processing system, the matching degree determination unit may determine the matching degree further based on the imaging data. In the information processing system, the similarity determination unit may determine a similarity of the movements of the multiple participants in the live performance based on a similarity of the movements of the multiple participants identified by analyzing the imaging data and a similarity of the movements of the multiple participants identified based on at least one of the motion data and the intensity of each of the multiple participants.

[0007] Any of the information processing systems may include a vital data acquisition unit that acquires vital data indicating vitals of the participants detected by the devices. In the information processing system, the intensity determination unit may determine the intensity of each of the participants based on the plurality of motion data and the plurality of vital data of the participants acquired by the vital data acquisition unit. In the information processing system, the intensity determination unit may determine the intensity of each of the participants by applying different weights to the motion data and the vital data of the participant and summing them.

[0008] According to one embodiment of the present invention, there is provided a program that may cause a computer to function as the information processing system.

[0009] According to one embodiment of the present invention, there is provided an information processing method executed by a computer. The information processing method may include a motion data acquisition step of acquiring motion data related to the motions of participants participating in a live event, the motion data being determined based on body motion data related to the body motions of the participants detected by a device worn by the participants. The information processing method may include an intensity determination step of determining an intensity level indicating a degree of intensity of the live event for each of the multiple participants, based on the multiple motion data of the multiple participants acquired in the motion data acquisition step. The information processing method may include an output data generation step of generating output data based on the intensity levels of the multiple participants.

[0010] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0011] [Figure 1] 1 illustrates a schematic diagram of an example of an information processing system 10. [Diagram 2] 2 is an explanatory diagram for explaining output data generated by the server 100. FIG. [Diagram 3] 1 illustrates an example of a data flow in an information processing system 10. [Figure 4] 2 illustrates another example of the data flow in the information processing system 10. [Diagram 5] 2 shows a schematic diagram of an example of projection image data displayed on a display 300. [Figure 6] 2 shows another example of the projection image data displayed on the display 300. [Figure 7] 1 illustrates an example of a processing flow of the server 100. [Figure 8] 2 illustrates an example of a functional configuration of the server 100. [Figure 9] 5 shows an example of learning data 502 used by the learning execution unit 108 for learning. [Figure 10] 10 is an explanatory diagram for explaining a coincidence determination performed by a coincidence determination unit 116. FIG. [Figure 11] An example of a hardware configuration of a computer 1200 functioning as the server 100 or the device 200 is illustrated in schematic form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0013] In so-called live events such as live entertainment and live events, the excitement of the participants is often important, and it is desirable to provide a mechanism that allows the participants to become more excited. The information processing system 10 according to the present embodiment, for example, quantifies the excitement of a live event. For example, by quantifying the excitement of the participants participating in one live venue, something like an individual competition of excitement can be realized. By each participant having the desire to aim for the top of the individual competition, each participant will consciously or unconsciously become excited, which can contribute to the excitement of the entire live venue. In addition, for example, when a live tour is carried out, by quantifying the excitement of each venue, the participants of each venue will have a competitive desire against the other venues, and the participants of each venue will consciously or unconsciously become excited, which can contribute to the excitement of each venue. The information processing system 10 can quantify the excitement of a live event by calculating the intensity level indicating the degree of intensity of the participants in the live event or by calculating the degree of agreement of the participants in the live event.

[0014] 1 illustrates an example of an information processing system 10. The information processing system 10 includes a server 100. The information processing system 10 may include a plurality of devices 200. The information processing system 10 may include a display 300. The information processing system 10 may include a camera 400.

[0015] The server 100 communicates with a plurality of devices 200, a display 300, and a camera 400. The server 100 communicates with the plurality of devices 200, the display 300, and the camera 400 via a network 30, for example.

[0016] The network 30 may include a mobile communication network. The mobile communication network may be compliant with a 5G (5th Generation) communication system. The mobile communication network may be compliant with an LTE (Long Term Evolution) communication system. The mobile communication network may be compliant with a 3G (3rd Generation) communication system. The mobile communication network may be compliant with a 6G (6th Generation) communication system or later mobile communication system. The network 30 may include a LAN (Local Area Network). The network 30 may include the Internet. The network 30 may include a cloud.

[0017] The device 200 may be any device that can be worn by the person 20 and can detect data related to the person 20. Wearing the device by the person 20 may include being placed in clothing worn by the person 20, being held by the person 20 in the hand, etc. The device 200 is, for example, a smartphone. The device 200 is, for example, a wearable device such as a wristwatch type device or a glasses type device.

[0018] The device 200, for example, detects the body movement of the person 20 to obtain body movement data related to the body movement of the person 20. The body movement data may include the acceleration of the person 20. The device 200 may be equipped with an acceleration sensor to detect the acceleration of the person 20. The body movement data may detect the angular velocity of the person 20. The device 200 may be equipped with an angular velocity sensor to detect the angular velocity of the person 20. The body movement data may include vibrations of the person 20. The device 200 may be equipped with a vibration sensor to detect vibrations of the person 20. These are merely examples, and the body movement data may include other things, and the device 200 may obtain the body movement data in a manner other than the examples given here.

[0019] The device 200, for example, detects the vital signs of the person 20 and acquires vital data indicating the vital signs of the person 20. The vital data may include a blood oxygen concentration of the person 20. The device 200 may detect the blood oxygen concentration of the person 20 by including a blood oxygen concentration sensor. The vital data may include a heartbeat of the person 20. The device 200 may detect the heartbeat of the person 20 by including a heartbeat sensor. The vital data may include a skin temperature of the person 20. The device 200 may detect the skin temperature of the person 20 by including a skin temperature sensor. The vital data may include a blood pressure of the person 20. The device 200 may detect the blood pressure of the person 20 by including a blood pressure sensor. These are merely examples, and the vital data may include other things, and the device 200 may acquire the vital data in a manner other than the methods exemplified here.

[0020] The device 200 may transmit the body movement data and the vital data to the server 100. If the device 200 has a communication function via the network 30, the device 200 may transmit the body movement data and the vital data to the server 100. If the device 200 is a wearable device or the like and does not have a communication function via the network 30, the device 200 may transmit the body movement data and the vital data to a smartphone or the like carried by the person 20, and the smartphone may transmit the body movement data and the vital data to the server 100.

[0021] The display 300 is installed, for example, at a live venue. The display 300 is installed at the live venue in a manner that allows viewing by a plurality of people 20 at the live venue. Although one display 300 is illustrated in Fig. 1, the information processing system 10 may include a plurality of displays 300 installed at the live venue.

[0022] The camera 400 is installed, for example, at a live concert venue. The camera 400 captures images of a plurality of people 20 in the venue. Although one camera 400 is illustrated in FIG. 1, the information processing system 10 may include a plurality of cameras 400 installed at the live concert venue.

[0023] The information processing system 10 according to the present embodiment acquires motion data related to the motion of the person 20 based on the body motion data acquired by the device 200. The motion data may include the type of motion. Examples of the type of motion include, but are not limited to, jumping, shaking head, swinging arms, and clapping. The motion data may include the amount of motion. For example, if the type of motion is jumping, the amount of motion may be the number of jumps. The motion data may include the magnitude of motion. For example, if the type of motion is jumping, the magnitude of motion may be the magnitude of the jump, for example, a large jump, a medium jump, and a small jump. The server 100 counts the number of times each type of motion is performed for each predetermined period, for example.

[0024] The process of acquiring the motion data based on the body movement data may be executed by the server 100. In this case, the server 100 receives the body movement data from the device 200 via the network 30. The process of acquiring the motion data based on the body movement data may be executed by the device 200. The device 200 acquires the motion data and communicates with the server 100 using, for example, an application. The application used by the device 200 may be a Web application, a native application, or a hybrid application that combines a Web application and a native application. When executed by the device 200, the server 100 receives the motion data from the device 200 via the network 30. The server 100 may receive both the motion data and the body movement data from the device 200.

[0025] The server 100 may use the motion data to determine an index indicating the excitement of the live performance. For example, the server 100 determines an intensity level indicating the intensity of the live performance for each of the multiple people 20 based on a plurality of motion data of the multiple people 20 participating in the live performance. The server 100 may determine the intensity level of each of the multiple people 20 based on a plurality of motion data of the multiple people 20 and a plurality of vital data of the multiple people 20. The server 100 may generate output data based on the intensity levels of the multiple people 20, and display the output data on the display 300 or distribute the output data via the network 30. This allows something like an exciting individual competition to be realized.

[0026] Also, for example, the server 100 determines the degree of matching of multiple participants in a live performance based on at least one of the motion data of each of the multiple people 20 and the intensity of each of the multiple people 20. The server 100 may determine the degree of matching further based on image data captured by the camera 400 while the multiple people 20 are participating in the live performance. The server 100 may generate matching data based on the determined degree of matching, and display the data on the display 300 or distribute the data via the network 30. This makes it possible to realize something like a team competition to create excitement at each of multiple live performance venues.

[0027] The server 100 may determine the degree of coincidence based on the imaging data captured by the camera 400 while the multiple people 20 are participating in the live performance. The server 100 may determine the degree of coincidence by analyzing the imaging data to determine the degree of coincidence of the movements of the entire venue. The server 100 may determine the degree of coincidence based on at least one of the motion data of each of the multiple people 20 and the intensity of each of the multiple people 20, and the imaging data. For example, when the server 100 determines that the movements of the multiple people 20 match based on at least one of the motion data and the intensity, it verifies the determination based on the analysis result of the imaging data. Note that the camera 400 may determine the degree of coincidence of the entire venue based on the imaging data and transmit the determined degree of coincidence to the server 100. In this case, the server 100 may adopt the degree of coincidence received from the camera 400, or may calculate the final degree of coincidence from the degree of coincidence determined by the server 100 itself and the degree of coincidence received from the camera 400.

[0028] The information processing system 10 may target an offline live performance performed on a stage 40 or the like at a live venue. Examples of the types of live performance include, but are not limited to, live music, musicals, plays, and comedy. The information processing system 10 may target a digital live performance projected on a screen or displayed on a display at a live venue. The performers may be virtual characters or the like.

[0029] The information processing system 10 may target an online live event where participants can participate at any location without going to a venue. In this case, the devices 200 of the multiple people 20 participating in the online live event transmit body movement data, vital data, motion data, etc. to the server 100 via the network 30.

[0030] 2 is an explanatory diagram for explaining output data generated by the server 100. Here, an example of output data 310 based on the intensity of a plurality of people 20 and an example of matching degree data 320 based on the matching degree are shown.

[0031] The output data 310 illustrated in FIG. 2 includes a gauge indicating the degree of intensity of an individual. The higher the intensity of the target person 20, the larger the gauge becomes. The server 100 generates, for example, output data 310 indicating the intensity of the person 20 in real time. The server 100 distributes, for example, the output data 310 to each of the multiple people 20. This allows each of the multiple people 20 to understand how excited he or she is determined to be. The server 100 may display the output data 310 of each of the multiple people 20 together on the display 300. At this time, the server 100 may display the registered name, number, etc. in association with the output data 310 so that the correspondence between the multiple people 20 and the output data 310 can be understood. This allows each of the multiple people 20 to understand the relative level of his or her excitement. The server 100 may generate output data 310 indicating the cumulative intensity of the person 20 in the live performance. This makes it possible to grasp the intensity of person 20 throughout the entire live performance.

[0032] The matching data 320 illustrated in FIG. 2 includes a gauge indicating the degree of matching. The higher the matching, the larger the gauge. The server 100 generates, for example, matching data 320 indicating the degree of matching in real time. The server 100 may distribute the matching data 320 to each of the multiple people 20 or display it on the display 300. This allows the multiple people 20 to understand the overall excitement of the venue in which they are participating. The server 100 may compile the matching data 320 for each of the multiple venues and display it on the display 300. At this time, the server 100 may display the venue name, number, etc. in association with the matching data 320 so that the correspondence between the venue and the matching data 320 can be understood. This allows the degree of matching for each venue to be understood. The server 100 may generate matching data 320 indicating the accumulation of the matching during the live performance.

[0033] 3 shows an example of a data flow in the information processing system 10. Here, an example will be described in which the server 100 distributes projection video data to the device 200 and the display 300 via the video distribution server 250. The server 100 and the video distribution server 250 may communicate with each other via the network 30. Note that the server 100 may distribute the projection video data to the device 200 and the display 300 without the intervention of the video distribution server 250.

[0034] 3, the device 200 acquires motion data based on the body movement data and transmits the motion data to the server 100. The server 100 may receive motion data from a plurality of devices 200.

[0035] The server 100 collects the motion data for each individual. The server 100 may collect the motion data for each timestamp. The server 100 may calculate the sum of the motions of each individual at the live venue. The server 100 generates output data based on the collected results. The server 100 transmits the generated output data to the video distribution server 250.

[0036] The video distribution server 250 generates projection video data from the received output data and transmits it to the display 300 and the device 200.

[0037] Fig. 4 shows an outline of another example of the data flow of the information processing system 10. In the example shown in Fig. 4, the camera 400 determines the degree of coincidence of the venue in the live performance based on the imaging data capturing the venue, and transmits the determined degree of coincidence to the server 100. The server 100 may receive the degree of coincidence data from the cameras 400 in a plurality of venues. Alternatively, the camera 400 may transmit the imaging data to the server 100, and the server 100 may determine the degree of coincidence of the venue in the live performance based on the imaging data.

[0038] The server 100 collects the coincidence data for each venue. The server 100 generates output data based on the collected data. The server 100 transmits the generated output data to the video distribution server 250.

[0039] The video distribution server 250 generates projection video data from the received output data and transmits it to the display 300.

[0040] Fig. 5 shows an example of the projected image data displayed on the display 300. The projected image data shown in Fig. 5 shows a gauge indicating the cumulative intensity value of the winner, and the location of the winner in the venue, with the person 20 with the highest cumulative intensity value at the end of the live performance as the winner. In Fig. 5, the participant in the third seat from the left in the front row wins the individual excitement competition, based on the amount of excitement gauge displayed.

[0041] Fig. 6 shows another example of the projection video data displayed on the display 300. The projection video data shown in Fig. 6 shows a gauge indicating the cumulative degree of coincidence of each venue and a captured image of each venue, with the venue with the highest cumulative degree of coincidence at the end of the live performance being declared the winning venue.

[0042] 7 shows an example of a processing flow of the server 100. In step (step may be abbreviated to S) 102, the server 100 acquires data for determining an index showing the excitement of the live performance. The server 100 acquires, for example, body movement data. The server 100 acquires, for example, vital sign data. The server 100 acquires, for example, motion data. The server 100 acquires, for example, imaging data from the camera 400. The server 100 acquires, from the camera 400, matching degree data identified by the camera 400.

[0043] In S104, the server 100 determines whether or not data acquisition is complete. The server 100 acquires data at predetermined intervals. For example, when acquiring data from the start to the end of a live performance to generate output data corresponding to the entire live performance, the server 100 determines that data acquisition is complete when the live performance ends. For example, when output data is generated periodically during the live performance, data acquisition is performed in accordance with the timing of generating the output data. In this case, it is determined that data acquisition is complete when a predetermined time has elapsed.

[0044] In S106, the server 100 identifies the actions of each of the multiple people 20 participating in the live performance. The server 100 identifies the actions of each of the multiple people 20 in a predetermined time unit, such as in units of seconds or minutes.

[0045] In S108, the server 100 determines an index indicating the excitement of the live performance. The server 100, for example, determines the intensity level of each of the multiple people 20. The server 100, for example, determines the degree of similarity of the multiple people 20 in the live performance.

[0046] In S110, the server 100 generates output data based on the index indicating the excitement determined in S108. The server 100 may display the generated output data on the display 300, distribute it to the device 200, or transmit it to the video distribution server 250.

[0047] 8 shows an example of a schematic functional configuration of the server 100. The server 100 may include a storage unit 102, a motion data acquisition unit 104, a motion identification unit 106, a learning execution unit 108, a vital data acquisition unit 110, an image data acquisition unit 112, an intensity determination unit 114, a matching determination unit 116, and an output data generation unit 118. Note that it is not essential that the server 100 has all of these components. For example, some of the components may be provided in the device 200.

[0048] The storage unit 102 stores various data. For example, the storage unit 102 stores body movement data received from the device 200. For example, the storage unit 102 stores motion data received from the device 200. For example, the storage unit 102 stores a learning model that receives body movement data as an input and outputs a type of motion.

[0049] The motion data acquisition unit 104 acquires motion data related to the motions of the participants participating in the live event. The motion data acquisition unit 104 stores the acquired motion data in the storage unit 102. The motion data acquisition unit 104 acquires the motion data from, for example, a device worn by the participant.

[0050] The motion identification unit 106 identifies the motion of a participant participating in a live performance from the body motion data detected by the device 200 worn by the participant. The motion identification unit 106 identifies the motion of the participant, for example, by inputting the body motion data acquired from the device 200 to a learning model stored in the storage unit 102, which uses the body motion data as input and outputs the type of motion. The motion data acquisition unit 104 may acquire motion data related to the motion of the participant identified by the motion identification unit 106.

[0051] The learning model may be generated by another device. In this case, the server 100 receives the learning model from the other device in advance and stores it in the storage unit 102. The learning model may be generated by the server 100. In this case, the storage unit 102 stores body movement data related to the body movement of a person detected by the device 200 when the person wearing the device 200 performs an action and the type of the action. The learning execution unit 108 generates the learning model by performing machine learning using the body movement data and the type of the action as learning data. The learning execution unit 108 stores the generated learning model in the storage unit 102. For example, by performing machine learning using the detection result by the acceleration sensor when a person jumps, a learning model capable of determining whether or not a person has jumped can be generated from the detection result by the acceleration sensor. Note that the learning execution unit 108 may set a threshold value for counting as a jump by performing machine learning using the detection result by the acceleration sensor when a person jumps. In this case, the motion identifying section 106 may detect that the person has jumped by comparing the detection result by the acceleration sensor, which is included in the body movement data, with the threshold value.

[0052] The motion identifying section 106 may identify the type of motion of the participant. The motion identifying section 106 may identify the amount of motion of the participant. The motion identifying section 106 may identify the magnitude of motion of the participant. The motion data acquiring section 104 may acquire motion data including the type, amount, and magnitude of motion of the participant.

[0053] The vital data acquiring unit 110 acquires vital data indicating the vital signs of the participant detected by the device 200. The vital data acquiring unit 110 causes the memory unit 102 to store the acquired vital data.

[0054] The imaging data acquisition unit 112 acquires imaging data of participants captured while participating in a live performance. For example, the imaging data acquisition unit 112 acquires imaging data of a plurality of participants captured by a camera 400 installed at a live performance venue from the camera 400. For example, the imaging data acquisition unit 112 acquires imaging data of participants participating in a live performance at an arbitrary location such as a home, captured by the camera 400 installed at the same location from the camera 400. The imaging data acquisition unit 112 stores the acquired imaging data in the storage unit 102.

[0055] The intensity level determination unit 114 determines the intensity level of each of the multiple participants based on the multiple motion data of the multiple participants acquired by the motion data acquisition unit 104. The intensity level determination unit 114 may continuously determine the intensity level of each of the multiple participants during a live event.

[0056] The intensity level determination unit 114 may determine the intensity level of each of the multiple participants based on the multiple pieces of motion data and the multiple pieces of vital data of the multiple participants acquired by the vital data acquisition unit 110. By using the vital data in addition to the motion data, the intensity level of the participants can be determined with higher accuracy.

[0057] The intensity determination unit 114 may determine the intensity of a participant by applying different weights to the motion data and vital data of the participant and summing them up. This can further improve the accuracy of the intensity. For example, the intensity determination unit 114 determines points for each type of motion and each type of vital data, and calculates the intensity by applying different coefficients to each and summing them up. As a specific example, the intensity determination unit 114 calculates the intensity y using the following formula 1.

[0058]

number

[0059] For example, X1 indicates the jump point, X2 indicates the arm swing point, and X3 indicates the blood oxygen concentration. The intensity determination unit 114 applies a coefficient to each of these points and then adds them up to calculate the intensity y.

[0060] The matching degree determination unit 116 determines the degree of matching between multiple participants in a live performance. The matching degree determination unit 116 may determine the degree of matching based on at least one of the motion data of each of the multiple participants acquired by the motion data acquisition unit 104 and the intensity of each of the multiple participants determined by the intensity determination unit 114. The matching degree determination unit 116 may determine the degree of matching based on at least one of the type of motion and timing of motion of each of the multiple participants and the intensity value and transition of the intensity of each of the multiple participants.

[0061] For example, the matching degree determination unit 116 increases the matching degree when a plurality of participants perform the same action at the same timing. The matching degree determination unit 116 may further increase the matching degree when more participants perform the same action at the same timing. As a specific example, the matching degree determination unit 116 adds 10 points to the matching degree when 10 participants jump at the same timing, and adds 100 points to the matching degree when 100 participants jump at the same timing. The matching degree determination unit 116 may change the points to be added depending on the type of action. For example, the matching degree determination unit 116 adds 10 points to the matching degree when 10 participants jump at the same timing, and adds 5 points to the matching degree when 10 participants make the same arm movement at the same timing.

[0062] For example, the matching degree determination unit 116 increases the matching degree as the intensity transitions of a plurality of participants match. The matching degree determination unit 116 increases the matching degree as more participants' intensity transitions match. For example, a larger value is added to the matching degree when it is determined that the intensity transitions of 100 participants match compared to when it is determined that the intensity transitions of 10 participants match. The matching degree determination unit 116 may also determine the matching degree of the intensity transitions, and increase the matching degree as the matching degree of the transitions becomes greater.

[0063] The matching degree determination unit 116 may determine the matching degree based on the imaging data acquired by the imaging data acquisition unit 112 in addition to at least one of the motion data of each of the participants acquired by the motion data acquisition unit 104 and the intensity of each of the participants determined by the intensity determination unit 114. For example, the matching degree determination unit 116 determines the matching degree of the multiple participants in a live performance based on the matching degree of the motions of the multiple participants identified by analyzing the imaging data and the matching degree of the motions of the multiple participants identified based on at least one of the motion data and intensity of each of the multiple participants. As a specific example, the matching degree determination unit 116 determines the matching degree of the multiple participants in a live performance by applying different weights to the matching degree of the motions of the multiple participants identified by analyzing the imaging data and the matching degree of the motions of the multiple participants identified based on at least one of the motion data and intensity of each of the multiple participants. The similarity determination unit 116 may determine the similarity of multiple participants in a live performance by correcting the movements of the multiple participants identified based on at least one of the motion data and the intensity of each of the multiple participants with the movements of the multiple participants identified by analyzing the imaging data.

[0064] The output data generating unit 118 may generate output data based on the intensity levels of the multiple participants. For example, the output data generating unit 118 may generate real-time data indicating the intensity levels continuously determined during a live broadcast for at least some of the multiple participants in real time. For example, the output data generating unit 118 may generate aggregate data based on an aggregate result of aggregating the intensity levels of the multiple participants.

[0065] The output data generating unit 118 may generate coincidence data based on the coincidence determined by the coincidence determining unit 116. For example, the output data generating unit 118 may generate coincidence data that displays the coincidence data of each of a plurality of venues in a comparative manner.

[0066] 9 is a schematic diagram showing an example of the learning data 502 used by the learning execution unit 108 for learning. The learning data 502 includes a plurality of detection results by an acceleration sensor when a person jumps. In the example shown in FIG. 9, the learning data 502 includes acceleration in the X-axis direction, the Y-axis direction, and the Z-axis direction when a person jumps. In this example, the axial directions may be expressed as "X" (front-to-back), "Y" (vertical), and "Z" (left-to-right) with respect to the front direction of the person 20.

[0067] The learning execution unit 108 may set thresholds for determining and counting a jump, such as X=0.3, Y=1.0, and Z=1.0, using the learning data 502. Furthermore, the learning execution unit 108 may execute machine learning using the learning data 502 to generate a learning model that outputs whether or not a person has jumped, based on the detection result of the acceleration sensor.

[0068] FIG. 10 is an explanatory diagram for explaining the coincidence determination by the coincidence determination unit 116. Here, an example will be described in which the coincidence is determined based on the jumping situations of multiple participants. In FIG. 10, only participant A, participant B, and participant C are shown for the sake of simplicity, but data for more participants may be used in practice. In the example shown in FIG. 10, the time slot is set to 1 second. The coincidence determination unit 116 determines the coincidence based on, for example, the proportion of times in the time slot that is equal to or greater than a predetermined threshold (e.g., 2, 3, etc.). The coincidence determination unit 116 may determine the maximum value of simultaneous actions on the time stamp as the coincidence.

[0069] 11 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the server 100 or the device 200. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the apparatus according to the above embodiment, or cause the computer 1200 to execute operations or one or more "parts" associated with the apparatus according to the above embodiment, and / or cause the computer 1200 to execute a process or steps of the process according to the above embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0070] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The computer 1200 also includes legacy input / output units such as a ROM 1230 and a keyboard 1242, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0071] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0072] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227 or the like and provides the programs or data to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0073] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0074] The programs are provided by a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information according to the use of the computer 1200.

[0075] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, the DVD-ROM 1227, or a recording medium such as an IC card, and transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0076] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0077] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0078] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0079] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0080] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0081] The computer readable instructions may include either assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0082] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0083] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0084] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that the process must be performed in this order. [Explanation of symbols]

[0085] 10 information processing system, 20 person, 30 network, 40 stage, 100 server, 102 memory unit, 104 motion data acquisition unit, 106 motion identification unit, 108 learning execution unit, 110 vital data acquisition unit, 112 imaging data acquisition unit, 114 intensity determination unit, 116 matching determination unit, 118 output data generation unit, 200 device, 250 video distribution server, 310 output data, 320 matching data, 400 camera, 502 learning data, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1226 DVD drive, 1227 DVD-ROM, 1230 ROM, 1240 input / output chip, 1242 Keyboard

Claims

1. a motion data acquisition unit that acquires motion data related to a motion of a participant participating in a live event, the motion data being acquired based on body motion data related to the body motion of the participant detected by a device worn by the participant; an intensity level determination unit that determines an intensity level indicating a degree of intensity of the live performance for each of the plurality of participants based on the plurality of motion data of the plurality of participants acquired by the motion data acquisition unit; an output data generation unit that generates output data based on the severity levels of the plurality of participants; An information processing system comprising:

2. a learning execution unit that generates a learning model by machine learning using body movement data related to the body movement of a person wearing the device when the person performs a movement and the type of the movement as learning data, the body movement data being input and the type of movement being output; a motion identification unit that inputs the body motion data detected by the device worn by the participant participating in the live performance into the learning model to identify the motion of the participant; Further equipped with The information processing system according to claim 1 , wherein the motion data acquisition unit acquires the motion data related to the motion of the participant identified by the motion identification unit.

3. The motion identification unit identifies a type, an amount, and a magnitude of the motion of the participant; The information processing system according to claim 2 , wherein the motion data acquisition unit acquires the motion data including a type, an amount, and a magnitude of the motion of the participant identified by the motion identification unit.

4. the intensity level determination unit continuously determines the intensity level of each of the plurality of participants during the live broadcast; The information processing system according to any one of claims 1 to 3, wherein the output data generation unit generates at least one of real-time data indicating the intensity level continuously determined during the live broadcast in real time for at least a portion of the plurality of participants, and aggregated data based on an aggregation result of aggregating the intensity levels of the plurality of participants.

5. a coincidence determination unit that determines a coincidence between the plurality of participants in the live event based on at least one of the motion data of each of the plurality of participants acquired by the motion data acquisition unit and the intensity of each of the plurality of participants determined by the intensity determination unit. Further equipped with The information processing system according to claim 1 , wherein the output data generating section generates coincidence data based on the coincidence.

6. The information processing system according to claim 5 , wherein the matching degree determination unit determines the matching degree based on at least one of the type of action and timing of the action of each of the multiple participants, and the intensity value and the transition of the intensity of each of the multiple participants.

7. An imaging data acquisition unit that acquires imaging data of the participants that are imaged while participating in the live show. Further equipped with The information processing system according to claim 5 , wherein the coincidence determination unit determines the coincidence further based on the imaging data.

8. The information processing system of claim 7, wherein the similarity determination unit determines a similarity of the movements of the multiple participants in the live event based on a similarity of the movements of the multiple participants identified by analyzing the imaging data and a similarity of the movements of the multiple participants identified based on at least one of the motion data and the intensity of each of the multiple participants.

9. a vital data acquisition unit that acquires vital data indicating the vital signs of the participant detected by the device; Further equipped with 4. The information processing system according to claim 1, wherein the intensity determination unit determines the intensity of each of the plurality of participants based on the plurality of motion data and the plurality of vital data of the plurality of participants acquired by the vital data acquisition unit.

10. The information processing system according to claim 9 , wherein the intensity determining unit determines the intensity of the participant by applying different weights to the motion data and the vital data of the participant and summing them.

11. A program for causing a computer to function as the information processing system according to any one of claims 1 to 3.

12. 1. A computer-implemented information processing method, comprising: A motion data acquisition step of acquiring motion data related to the motion of a participant participating in a live event, the motion data being identified based on body motion data related to the body motion of the participant detected by a device worn by the participant; an intensity level determination step of determining an intensity level indicating a degree of intensity of the live performance of each of the plurality of participants based on the plurality of motion data of the plurality of participants acquired in the motion data acquisition step; generating output data based on the intensity levels of the plurality of participants; An information processing method comprising:

Images