Information processing device, information processing method, and program

Abstract

In competitions in which comparison with others is important, such as race competitions, it is desirable to see the movement of the overall competition as well as the relationships between the movements of individual competitors all at once. To this end, this information processing device comprises a content generation unit that generates content for display by combining a first image which is captured by a first imaging device, and a second image which is captured by the first imaging device or another imaging device and in which a portion of a subject of the first image is set as a subject thereof.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and in particular to a technology for generating content by combining two images. [Background technology]

[0002] For example, when watching a sporting event, it is difficult to visually recognize the performances of multiple subjects at the same time. In response to this, Patent Document 1 discloses a method for generating one composite stream data by synthesizing stream data captured by multiple television cameras, and distributing the composite stream data via a network in response to a request from a viewer terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-18583 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in sports such as races where comparison with others is important, there is a demand to be able to see the relationship between the movements of the entire event and the movements of each individual athlete at once.

[0005] Therefore, an object of the present technology is to generate appropriate content for grasping the movement of a subject within the whole. [Means for solving the problem]

[0006] The information processing device of the present technology includes a content generation unit that generates content for display by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, the second image having a subject that is a part of the subject of the first image. When photographing a plurality of subjects, there is a demand for simultaneously recording and viewing images of each subject or images with different angles of view.

[0007] The information processing device described above may further include a synchronization processing unit that performs a synchronization process to synchronize the first image and the second image. This makes it possible to transmit display content that combines the first image and the second image to a user terminal or the like.

[0008] The above-mentioned information processing device may further include a synchronization processing unit that synchronizes the first image and the second image. This makes it possible to grasp the temporal relationship between the first and second images even if they are taken at different angles of view.

[0009] In the above information processing device, the display content may include overlay information. The superimposition information is, for example, character (text) information or image information.

[0010] In the above information processing device, the superimposition information may be explanatory information about the first image or the second image. The explanatory information is, for example, in the case of a race image, explanatory information that helps the viewer understand the image, such as the date and time of the race, the venue, the name of the race, and the names of the participants.

[0011] In the information processing device described above, the overlay information may be comparison information for a subject of the second image. The comparison information is, for example, information indicating a difference between the target record of the athlete who is the subject of the second image and the current record, specifically, information indicating how far ahead or behind the target record the athlete is.

[0012] In the information processing device described above, the comparison information may be information indicating a score for a subject of the second image. The performance of the subject of the second image is, for example, the past record of the athlete who is the subject of the second image.

[0013] In the information processing device described above, the comparison information may be information indicating a performance of a person other than the subject of the second image. The results of a person other than the subject of the second image may be, for example, the record of the best performer among other runners competing in the same race, or may be a Japanese record, a world record, or the like.

[0014] The above-mentioned information processing device may include an analysis processing unit that performs image analysis on the subject of the second image and extracts features that affect the grades of the subject of the second image, and the superimposition information may be information regarding the features. The feature amount that influences the performance of the subject of the second image is, for example, the number or frequency of breaths, or the number of times the subject strokes their hands in swimming.

[0015] The information processing device may further include an authentication processing unit that performs user authentication based on received user information, and the transmission processing unit may select and transmit the display content based on a result of the authentication. Display content is image content that includes personal information. Such content is sent only if user authentication is successful.

[0016] The information processing device may include a content management section that sets a disclosure range for each display content. There are cases where viewing of the display content needs to be restricted depending on the subject athlete, and cases where it is not.

[0017] The information processing device may further include an authentication processing unit that performs user authentication based on received user information, and the transmission processing unit may transmit the display content based on a result of the authentication and the disclosure range. For example, the user is identified by user authentication, and a decision is made as to whether or not to accept the request of the user based on the disclosure range set in the display content.

[0018] In the above information processing device, the synchronization process may include comparing the first image with the second image. For example, when the first and second images are moving images, the synchronization process is performed by comparing parts of both images and calculating the degree of similarity.

[0019] The transmission processing unit in the above-mentioned information processing device may perform streaming distribution. For example, during a competition, a first official image capturing the entire event from a bird's-eye view and a second official image focusing on a specific athlete will be combined and streamed.

[0020] The information processing method of the present technology is executed by an information processing device that generates content for display by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of the subject of the first image as the subject.

[0021] The program of the present technology causes an information processing device to execute a generation process of generating content for display by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a portion of the subject of the first image as the subject. This makes it possible to generate appropriate content for understanding the movement of the subject within the whole. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 illustrates a configuration of a distribution system according to an embodiment. [Diagram 2] FIG. 1 is a block diagram of an information processing apparatus according to an embodiment. [Diagram 3] FIG. 2 is a functional block diagram of a distribution server device according to an embodiment. [Figure 4] 4 is an example of a distribution content according to an embodiment. [Diagram 5] 11 is another example of distribution content according to the embodiment. [Figure 6] 13 is a flowchart illustrating an example of a distribution content generation process according to the embodiment. [Figure 7] 13 is a flowchart illustrating an example of a coaching information extraction process according to an embodiment. [Figure 8] 11 is a flowchart illustrating an example of a request handling process according to an embodiment. [Figure 9] 11 is a flowchart showing an example of a distribution content transmission process according to the embodiment; [Figure 10] 13 is a flowchart illustrating an example of a coaching information transmission process according to an embodiment. [Figure 11] 11 is a flowchart illustrating an example of a live distribution process according to an embodiment. [Figure 12] FIG. 11 is a diagram for explaining a first modified example of an editing example of captured image data according to an embodiment. [Figure 13] 11 is a flowchart showing an example of a distribution content generation request receiving process according to the embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The embodiments will be described below in the following order. <1. System configuration> 2. Configuration of distribution server device <3. Examples of editing captured image data> <4. Example of Processing in Distribution Server Device> <4-1. Distribution content generation process> <4-2. Coaching information extraction processing> <4-3. Request Response Processing> <5. Live Streaming> <6. Variations> <6-1. Modification 1 of the Editing Example of Captured Image Data> <6-2. Modification 2 of the Editing Example of Captured Image Data> <7. Summary> <8. This Technology>

[0024] <1. System configuration> The configuration of a distribution system 1 in this embodiment will be described with reference to Fig. 1. In the following description, a system that distributes images related to competitive swimming as a sporting event will be given as an example of the distribution system 1. The term "image" here includes both moving images and still images.

[0025] The distribution system 1 is configured by connecting a distribution server device 3, a team terminal 4, and an individual terminal 5 via a communication network 2 so that they can communicate with each other. The distribution server device 3 is also capable of communicating with a venue facility system 6. The venue facility system 6 is configured to include a plurality of imaging devices 7.

[0026] The communication network 2 may be, for example, the Internet, a home network, a LAN (Local Area Network), a satellite communication network, or any other type of network.

[0027] The distribution server device 3 is, for example, an information processing device managed by an organization that manages various swimming competitions and events, and is configured by a PC (Personal Computer), a workstation, or the like.

[0028] The distribution server device 3 manages schedules for swimming competitions and events, manages sporting events, and manages participants (athletes), etc. The distribution server device 3 transmits schedule information, sporting events, and information about participants to the team terminal 4 and the individual terminal 5. Furthermore, the distribution server device 3 receives a request to enter a tournament and executes an entry process for the tournament.

[0029] The distribution server device 3 distributes images captured by an imaging device 7 installed in the swimming venue to the team terminals 4 and the individual terminals 5. The distribution server device 3 is capable of communicating with the imaging device 7 provided in the venue facilities system 6.

[0030] The communication between the distribution server device 3 and the imaging devices 7 may be possible via a network or by direct connection. The venue facility system 6 may include a management terminal for centrally managing the imaging devices 7, and the distribution server device 3 may be able to communicate with the imaging devices 7 via the management terminal. Furthermore, the venue facility system 6 may be connected to the communication network 2 , and the distribution server device 3 may be capable of communicating with the imaging device 7 via the communication network 2 .

[0031] The distribution server device 3 generates content for distribution (hereinafter, referred to as "distribution content") using captured image data acquired from the imaging device 7. The distribution content is, for example, image data that combines captured image data acquired from a plurality of imaging devices 7, and may further include superimposed text information such as caption information and metadata, and image data. The distribution content is the content that is displayed on each information processing device, and therefore can also be called display content.

[0032] The distribution content generated by the distribution server device 3 is distributed to team terminals 4 and personal terminals 5 as appropriate.

[0033] The distribution server device 3 may be an information processing device that is temporarily installed in the venue during the tournament period. In this case, the distribution server device 3 may be configured by cooperation between a permanent information processing device owned by the organizing body that executes various processes that are constantly executed outside the tournament period, such as schedule management, player information management, and entry processing, and a temporary information processing device that is temporarily installed in the venue that executes various processes that are executed especially during the tournament period, such as control of the imaging device 7 and processing of captured image data.

[0034] In the following example, a description will be given of an example in which the distribution server device 3 is configured to include both a permanent information processing device and a temporary information processing device.

[0035] The team terminals 4 are information processing devices owned by the swimming team or club team, such as PCs or PDAs (Personal Digital Assistants). The team terminals 4 are available only to users (staff and athletes) who belong to the swimming team, and are capable of making various requests to the distribution server device 3. The various requests include, for example, a request to transmit distribution content and a request to transmit tournament information. Furthermore, the team terminal 4 is capable of transmitting a request to enter the tournament to the distribution server device 3 by specifying the tournament, the event, and the competitor.

[0036] The team terminal 4 manages information about the athletes who belong to the swimming team. The team terminal 4 is capable of transmitting results, participation information, and the like about the athletes who belong to the team in response to a request from the personal terminal 5, for example.

[0037] The personal terminal 5 is an information processing device owned by the athlete, and may be, for example, a portable terminal device such as a smartphone or tablet, a mobile phone, a PC, a game device, a video device, a PDA, or another device with a communication function.

[0038] The personal terminal 5 is capable of transmitting a request for obtaining tournament results and personal information to the team terminal 4 and obtaining the respective information. In addition, the personal terminal 5 may be configured to be capable of receiving tournament information and distribution content from the distribution server device 3. Furthermore, the personal terminal 5 may be capable of transmitting a login request to the team terminal 4 and the distribution server device 3. For example, an ID (Identification) and a password are transmitted as login information to each device in order to obtain various information, and authentication processing is performed in each device.

[0039] The venue facilities system 6 includes imaging devices 7 installed at the swimming competition venue. As described above, the venue facilities system 6 may be configured to include a management terminal for managing the imaging devices 7 in an integrated manner.

[0040] The imaging device 7 is a video camera or the like installed at the swimming venue, and is provided, for example, as an imaging device 7 that images the entire pool, an imaging device 7 that captures images of each lane, an imaging device 7 that captures the scoreboard, or an imaging device 7 that captures the spectator seats.

[0041] 2. Configuration of distribution server device The configuration of the distribution server device 3 will be described. As described above, here, an example will be described in which the distribution server device 3 has a management function for the imaging device 7 installed at the swimming venue. For example, the distribution server device 3 is composed of multiple information processing devices. Each of the information processing devices has a configuration as shown in FIG. A CPU 71 of the information processing device executes various processes according to a program stored in a ROM 72 or a program loaded from a storage unit 79 to a RAM 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes as appropriate. The CPU 71, the ROM 72, and the RAM 73 are interconnected via a bus 74. To this bus 74, an input / output interface 75 is also connected.

[0042] The input / output interface 75 is connected to an input unit 76 including an operator and an operating device. For example, the input unit 76 may be various types of operators or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, or a remote controller. An operation by the user is detected by the input unit 76, and a signal corresponding to the input operation is interpreted by the CPU 71.

[0043] Further, a display unit 77 formed of an LCD or an organic EL panel, etc., and an audio output unit 78 formed of a speaker, etc. are connected to the input / output interface 75 either integrally or separately. The display unit 77 is a display unit that performs various displays, and is configured, for example, by a display device provided in the housing of the distribution server device 3, a separate display device connected to the distribution server device 3, or the like. The display unit 77 executes display of images for various image processing, moving images to be processed, etc. on the display screen based on instructions from the CPU 71. Furthermore, the display unit 77 performs display of various operation menus, icons, messages, etc., that is, as a GUI (Graphical User Interface), based on instructions from the CPU 71.

[0044] The input / output interface 75 may be connected to a storage unit 79 configured with a hard disk or solid-state memory, or a communication unit 80 configured with a modem or the like.

[0045] The communication unit 80 performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, and the like. In this embodiment, the communication unit 80 has a function of communicating with the imaging device 7.

[0046] A drive 82 is also connected to the input / output interface 75 as required, and a removable recording medium 81 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is appropriately mounted thereon. By using the drive 82, various computer programs and the like can be read from the removable recording medium 81. The computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as necessary.

[0047] In this distribution server device 3, for example, software for the processing of the present disclosure can be installed via network communication by the communication unit 80 or via a removable recording medium 81. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, or the like.

[0048] For example, a functional configuration as shown in FIG. 3 is constructed in the CPU 71 of the distribution server device 3 by such software (application program). The distribution server device 3 is configured to have various functions such as a camera controller 11, a switcher 12, a player information management unit 13, a content management unit 14, an image editing processing unit 15, an analysis processing unit 16, a synchronization processing unit 17, an authentication processing unit 18 and a distribution control unit 19.

[0049] The camera controller 11 controls the imaging device 7 installed in the venue. Specifically, the camera controller 11 controls the imaging device 7 to rotate in the panning and tilting directions, to zoom in and out, and to change the quality of the captured image data, such as the image quality and color tone. The camera controller 11 can also control the transmission of captured image data captured by the imaging device 7.

[0050] The switcher 12 switches between images from the multiple imaging devices 7 to perform processes such as changing live streaming images and switching images to be displayed on a large display installed in the venue.

[0051] It should be noted that the camera controller 11 and the switcher 12 may each be realized as a separate device.

[0052] The player information management unit 13 manages various information related to players belonging to the governing body. For example, player information includes information for identifying an individual, such as a player's name, player ID, team information, address, contact information, date of birth, and age at the time of the tournament, as well as information on tournaments participated in and record information.

[0053] The content management unit 14 manages image content information generated for distribution. The image content is generated by combining captured image data acquired from multiple imaging devices 7, for example, and is distribution content for distribution to team terminals 4 and individual terminals 5, and is also display content displayed on each terminal. The content management unit 14 manages image content and metadata and caption information associated with the image content. The metadata and caption information include, for example, the subject name (athlete name) related to the image content, the name of the competition, the venue of the competition, the shooting date and time, the swimming event, the course number used by the athlete, the swimming record, the ranking, etc. In addition, information for identifying the imaging device 7 that captured the athlete, the shooting settings of the imaging device 7, etc. may be recorded.

[0054] The content management unit 14 sets and stores a disclosure range for each image content. The disclosure range may be, for example, "distribution only to the subject athlete," "distribution to the athlete's team," or "distribution to anyone."

[0055] The image editing processing unit 15 performs editing processing on the captured image data acquired from the imaging device 7. Various editing processing is possible, such as processing for synthesizing a plurality of captured image data acquired from a plurality of imaging devices 7 to generate one image data, processing for superimposing text information and images, and processing for cutting out an image within a predetermined range from the acquired captured image data.

[0056] Examples of text information to be superimposed include athlete information, tournament information, swimming records, etc. Examples of images to be superimposed include images for visually showing the past results of the athlete who is the subject of the acquired captured image data, images for visually showing record information of various tournaments and countries, etc.

[0057] The analysis processing unit 16 is a processing unit that performs image analysis on the captured image data acquired from the imaging device 7 to acquire coaching information. For example, in the case of competitive swimming, the number of times the swimmer takes a breath, the number of times the swimmer strokes the water, the number of times the swimmer kicks the water, and the like while swimming 50 m are acquired by image analysis. In addition, the position of the swimmer and the speed (swimming speed) at that time may be calculated by image analysis. These pieces of information are regarded as coaching information that can be used for coaching. In extracting the coaching information, for example, a deep learning method such as a Convolutional Neural Network (CNN) may be used.

[0058] The synchronization processing unit 17 performs processing for synchronizing a plurality of pieces of captured image data when a single piece of distribution content is generated by synthesizing a plurality of pieces of captured image data.

[0059] The authentication processing unit 18 performs authentication processing to determine whether or not a user is a member based on ID information and password information received from other terminals such as the team terminal 4 and the personal terminal 5. For users or terminals for which authentication processing has been successful, authentication processing unit 18 may be configured to transmit, for example, a two-dimensional barcode as information indicating that the user is a member. For example, a user who receives two-dimensional barcode information can smoothly enter the venue or the related-person area by holding the two-dimensional barcode over a reader or the like installed at the venue. Two-dimensional barcodes can also be used when obtaining permission to take photographs at the swimming venue.

[0060] The delivery control unit 19 obtains image content from the database and controls delivery of the image content to a terminal that has transmitted a delivery request for the image content. The delivery control unit 19 may also control delivery based on the public range of the image content.

[0061] The distribution control unit 19 may also control the live distribution of captured image data captured at the swimming competition venue in real time. For example, live images may be distributed to users who are family members visiting the swimming competition venue. In this case, different live images may be distributed to each user. Specifically, for example, captured image data (whole image) from the imaging device 7 capturing an overhead image of the entire pool and captured image data (close-up image, individual image) from the imaging device 7 capturing an individual athlete by zooming in may be synthesized in the image editing processing unit 15, and individual image content obtained for each athlete may be distributed to each user as a live image. This allows the user to receive image content created specifically for his or her child.

[0062] At least some of the various functions of the distribution server device 3 described above may be functions provided by cloud computing. Also, for example, instead of installing the distribution server device 3 at the swimming competition venue, the distribution server device 3 may be provided as a stationary server device owned by the organizing body, and the distribution server device 3 may have various functions.

[0063] <3. Examples of editing captured image data> An example of image content edited by image edit processing unit 15 of distribution server device 3 will be described. There are cases where a user wants to simultaneously view an image of a racer related to the user in a zoomed-in shot, an image of the entire race from a bird's-eye view, and information on the electronic scoreboard that displays records related to the race. To meet this need, the captured image data acquired from the imaging device 7 is edited to generate image content such as that shown in FIG.

[0064] FIG. 4 shows a snapshot of the image content generated for delivery to users. The image content is composed of multiple areas. For example, the first area AR1 is an area where an image for visually checking the progress of the entire race is displayed, and image data acquired from an imaging device 7 that captures an image of the entire pool is displayed. Hereinafter, image data generated by synthesizing multiple image data in this way will be referred to as "synthetic image data."

[0065] In the second area AR2, captured image data acquired from the imaging device 7 capturing close-up images of individual athletes is displayed. If the image of the entire pool has a sufficiently high resolution, a specific area of ​​an individual athlete in the image displayed in the first area AR1 may be cropped and enlarged and displayed in the second area AR2. That is, images may be displayed in the first area AR1 and the second area AR2 using image data acquired from one imaging device 7.

[0066] In Fig. 4, two areas are provided in which the captured image data is displayed, but of course this is just one example, and three or more areas may be provided. In that case, for example, as shown in Fig. 4, in addition to the first area AR1 and the second area AR2, a third area AR3 may be provided in which the captured image data acquired from the imaging device 7 that captured the electronic scoreboard is displayed. Moreover, the regions may be set to the same size or different sizes, and may be arranged so that one region overlaps with another region.

[0067] Instead of an image, text information may be displayed. An example is shown in Figure 5. The text information may be, for example, explanatory information for explaining the image. Figure 5 shows an example in which, in addition to a first area AR1 in which an image capturing an overall bird's-eye view is displayed and a second area AR2 in which a close-up image of each individual player is displayed, a third area AR3 is provided for displaying text information extracted from the information displayed on the scoreboard.

[0068] The third area AR3 is an area that is displayed in an overlapping manner on the second area AR2. The text information to be displayed as the explanatory information in the third area AR3 may be obtained, for example, by performing image analysis on the captured image data of the electronic bulletin board. Character recognition is performed on the captured image data of the electronic bulletin board to obtain the text information. Alternatively, the text information transmitted for display on the electronic bulletin board may be obtained, and the text information to be displayed as the explanatory information may be extracted from the text information.

[0069] In addition, information on participants taking part in the competition may be obtained from a database, and time information may be obtained from a device that manages progress information for the swimming race (for example, lap time information for every 50 m), and text information may be generated from the obtained information. Other text information that may be obtained includes the name of the tournament, the location of the tournament, the date and time of the photo, the swimming event, the name of the athlete displayed in the second area AR2, the course number used by the athlete, swimming records, rankings, and personal best information.

[0070] When displaying text information, the composite image data and the text information shown in Fig. 4 may be separated and transmitted to another terminal such as the team terminal 4 or the personal terminal 5, and the text information may be superimposed on the composite image data and displayed by processing of the other terminal. In this case, it is easy to allow the user to freely set the position at which the text information is superimposed and to set the display and non-display of the text information.

[0071] Alternatively, the text information may be incorporated in advance as image information into the composite image data by the image editing processing unit 15 of the distribution server device 3, and the combined image data may be transmitted to another terminal. Also, the text information may be displayed without providing the third area AR3. For example, by incorporating the text information in the captured image data to be displayed in the second area AR2, it becomes unnecessary to provide the third area AR3 for displaying the text information.

[0072] <4. Example of Processing in Distribution Server Device> <4-1. Distribution content generation process> Various processes executed by the distribution server device 3 will now be described. First, the process of generating distribution content will be described with reference to Fig. 6. The distribution content generation process is a process executed after a swimming competition, for example, and is a process of generating individual distribution content for each swimmer.

[0073] In the distribution content generation process, the distribution server device 3 selects one athlete in step S101. The distribution content generation process is a process that is executed after the end of a tournament, for example, using captured image data acquired from multiple imaging devices 7. Therefore, in step S101, one athlete is selected from among the athletes who participated in the tournament. Note that if there is an athlete who competes in multiple races in one tournament, for example, if there is an athlete who competes in both the preliminary and final rounds, the process of selecting both the race and the athlete is executed in step S101.

[0074] In step S102, the distribution server device 3 executes a process of selecting an overall image. Specifically, the captured image data acquired from the imaging device 7 capturing an overall bird's-eye view is selected. The captured image data to be acquired is of a specific race in which the runner selected in step S101 participated.

[0075] In step S103, the distribution server device 3 executes a process of selecting an individual image of the athlete. This process is for captured image data of the athlete identified in step S101. In the case of an athlete participating in multiple races, one captured image data is selected based on the race information selected in step S101.

[0076] In step S104, the distribution server device 3 acquires text information. This process is performed, for example, to acquire text information to be displayed in the third area AR3 in Fig. 5. The acquired information includes, for example, swimming venue information, tournament name, race information, course number used by the athlete during the race, athlete name, performance information in the race such as lap time, personal best time, etc. If text information is not used, the process of step 104 does not need to be executed.

[0077] In step S105, the distribution server device 3 performs a synthesis process on the captured image data. Through this process, the distribution content shown in Fig. 4 and Fig. 5 is generated.

[0078] The generated distribution content is stored in the storage unit 79 or the like by the distribution server device 3 executing the process of step S106.

[0079] In step S107, the distribution server device 3 determines whether the above-mentioned various processes have been completed for all the athletes who participated in the tournament. If there are athletes who have not yet been processed, the distribution server device 3 returns to the process of step S101. On the other hand, if there is no unprocessed player, the distribution server device 3 ends the series of processes shown in FIG.

[0080] By executing the distribution content generation process shown in FIG. 6, different distribution content is generated for each player who participates in the tournament. Although an example has been given in which the process shown in Fig. 6 is executed for all athletes who participated in the tournament, it may be executed for some athletes. It may also be executed appropriately when a distribution request for distribution content is received from a team terminal 4 or an individual terminal 5. When the distribution content generation process shown in Fig. 6 is executed in response to reception of a distribution request, unnecessary processing does not need to be executed, thereby reducing the processing burden on the distribution server device 3.

[0081] <4-2. Coaching information extraction processing> The coaching information extraction process executed by the distribution server device 3 will be described with reference to Fig. 7. The coaching information extraction process is a process for extracting information used for coaching by analyzing captured image data acquired from the imaging device 7. The coaching information extraction process is a process that is executed after a tournament, for example.

[0082] In step S201, the distribution server device 3 selects one player. This process is the same as the process in step S101 in Fig. 6, and is a process of selecting one player who has participated in the tournament.

[0083] In step S202, the distribution server device 3 selects individual images of players. This process is similar to the process in step S102 in FIG.

[0084] In step S203, the distribution server device 3 performs image analysis processing. The image analysis processing is processing for analyzing the captured image data selected in step S202 and extracting feature amounts. In the analysis, for example, a deep learning method such as CNN may be used.

[0085] In step S204, the distribution server device 3 extracts coaching information. This process extracts the number of breaths, the number of strokes, the number of kicks, etc. as coaching information based on the analysis results of the previous image analysis process. In addition, the change in swimming speed from the start to the end of the race may also be extracted.

[0086] In step S205, the distribution server device 3 executes a process of storing the coaching information. Specifically, the coaching information is stored in association with a competition, race, or the like in which the athlete participated. In step S206, the distribution server device 3 determines whether the above-mentioned various processes have been completed for all the athletes who participated in the tournament. If there are athletes who have not yet been processed, the distribution server device 3 returns to the process of step S201. On the other hand, if there is no unprocessed player, the distribution server device 3 ends the series of processes shown in FIG.

[0087] By executing the coaching information extraction process shown in FIG. 7, coaching information for each player who participated in the tournament is extracted. 7 is executed for all the players who participated in the tournament, it may be executed for some of the players. It may also be executed appropriately when a coaching information transmission request is received from the team terminal 4 or the personal terminal 5. When the coaching information extraction process shown in FIG. 7 is executed in response to receiving a transmission request, unnecessary processing does not need to be executed, and therefore the processing load on the distribution server device 3 is reduced.

[0088] The coaching information extracted here may be displayed as text information in the third area AR3 shown in FIG.

[0089] <4-3. Request Response Processing> The distribution server device 3 receives various requests from the team terminals 4 and the personal terminals 5. Therefore, the distribution server device 3 confirms receipt of various requests and executes a process corresponding to each request. An example of a request handling process executed by the distribution server device 3 is shown in FIG.

[0090] In step S301, the distribution server device 3 determines whether or not a member registration request has been received. If it is determined that a member registration request has been received, the distribution server device 3 executes a member registration process in step S302 as a corresponding process.

[0091] In the member registration process, for example, web page data for member registration is transmitted to the other information processing device that has transmitted the request, whereby the web page for member registration is displayed on the other information processing device. In addition, the member registration process receives member information entered on another information processing device, checks whether the entered information is correct, prompts the user to re-enter the information if there is an error, and registers new member information in the member database if there is no problem with the entered information. The member registration process also includes a process of issuing an ID and password for logging in to enable use of various services.

[0092] After executing the process of step S302, the distribution server device 3 returns to the process of step S301 again.

[0093] If it is determined in step S301 that a membership registration request has not been received, distribution server device 3 determines in step S303 whether or not a tournament entry request has been received. If it is determined that a tournament entry request has been received, distribution server device 3 executes a tournament entry process in step S304 as the corresponding process.

[0094] In the tournament entry process, for example, web page data for applying for tournament entry is transmitted to the other information processing devices that received the request, causing the other information processing devices to display a web page for applying for tournament entry.

[0095] On the web page for submitting a tournament entry application, for example, operations such as selecting the tournament to enter and inputting the name and player ID of the player to be entered are possible. The distribution server device 3 receives each piece of input information and completes the tournament entry by registering it in a database that manages the players participating in the tournament.

[0096] If entry to the tournament is not possible due to results or the like, web page data for notifying the other information processing devices of the fact that entry was not possible is transmitted to the other information processing devices.

[0097] After executing the process of step S304, the distribution server device 3 returns to the process of step S301 again.

[0098] If it is determined in step S303 that a tournament entry request has not been received, the distribution server device 3 determines in step S305 whether or not a distribution content transmission request has been received. If it is determined that a distribution content transmission request has been received, the distribution server device 3 executes a distribution content transmission process in step S306 as a corresponding process.

[0099] An example of the distribution content transmission process will be described with reference to FIG. In step S401, the distribution server device 3 determines whether or not the user who has requested the transmission of the distribution content is in a logged-in state.

[0100] If the user is not logged in, the distribution server device 3 executes a login process in step S402. In the login process, an ID and password are received from another information processing device used by the user who made the request, and are collated with the information stored in a login database to determine whether or not the login is permitted. The result of the request is transmitted to the other information processing apparatus, which then displays information indicating whether the login was successful or not.

[0101] After the login process has been normally completed, or if it is determined in step S401 that the user is already in a logged-in state, the distribution server device 3 receives distribution content selection information in step S403. The distribution content selection information is information for selecting one distribution content from a plurality of distribution contents, and is, for example, information designated by the user regarding which athlete and which race in which tournament the user would like to receive distribution content.

[0102] After receiving the distribution content selection information, the distribution server device 3 executes a process of acquiring disclosure range information of the specified distribution content in step S404. The disclosure range information is, for example, information associated with the distribution content and stored in a database.

[0103] In step S405, the distribution server device 3 determines whether or not the designated distribution content may be transmitted. The determination of whether or not transmission is permitted is based on the requesting user who made the transmission request and the disclosure range information. For example, if the user who made the distribution request is the athlete himself, the requesting user is included in the disclosure range, and transmission of the distribution content is permitted. Furthermore, if the requesting user is a family member of the player and the disclosure scope is permitted to include the family, transmission of the distribution content is permitted. On the other hand, if the requesting user is a person unrelated to the player and the disclosure scope is limited to only the player himself or his team, the transmission of the distribution content is not permitted since the requesting user is not included in the disclosure scope. When the disclosure scope is set to all users, transmission of the distribution content is permitted regardless of the requesting user.

[0104] If the transmission of the distribution content is permitted, the distribution server device 3 transmits the distribution content in step S406. On the other hand, if the transmission of the distribution content is not permitted, the distribution server device 3 performs processing in step S407 to transmit to the information processing device that made the request that the transmission of the distribution content was not permitted.

[0105] Returning to the explanation of Figure 8. After executing the distribution content transmission process in step S306, the distribution server device 3 returns to the process in step S301, and determines whether or not various requests have been received.

[0106] If it is determined that a distribution content transmission request has not been received, the distribution server device 3 determines in step S307 whether or not a coaching information transmission request has been received. If it is determined that a coaching information transmission request has been received, the distribution server device 3 executes a coaching information transmission process in step S308 as a corresponding process.

[0107] An example of the coaching information transmission process will be described with reference to FIG. In step S501, the distribution server device 3 determines whether the user who has requested the transmission of coaching information is logged in, and executes login processing in step S502 as necessary. These processing steps are similar to the processing steps S401 and S402 in FIG.

[0108] If the login process is normally completed, or if it is determined in step S501 that the user is already logged in, the distribution server device 3 receives coaching information selection information for selecting a type of coaching information in step S503. This information is information for specifying what type of coaching information the user desires, and is at least information that can specify which performance is the target, such as the name of the tournament, the name of the race, or the name of the athlete.

[0109] The coaching information selection information may also include information for identifying which item the user selected from among items such as the position at which the swimmer entered the water at the start, the number of times they took a breath, the number of times they stroked the water, the number of times they kicked the water, the timing at which they began to reach out for a turn, the starting position of a turn, and changes in swimming speed.

[0110] By receiving this coaching information selection information, the distribution server device 3 can know which of a plurality of pieces of coaching information should be transmitted.

[0111] In step S504, the distribution server device 3 acquires coaching information based on the information received in step S503. Next, in step S505, the distribution server device 3 transmits the coaching information.

[0112] Note that a disclosure range may be set for the coaching information. In that case, after step S503, a process for determining whether or not the identified coaching information may be transmitted, for example, the processes of steps S404 and S405 in Fig. 9, may be executed. Then, only when it is determined that the information may be transmitted, the processes of steps S504 and S505 may be executed.

[0113] Returning to the explanation of Figure 8. After executing the coaching information transmission process in step S308, the distribution server device 3 returns to the process in step S301.

[0114] If the distribution server device 3 determines that it has not received a coaching information transmission request, it determines in step S309 whether it has received an entry information request. The entry information request is made, for example, by an athlete who plans to participate in a tournament, and requests the transmission of various information about the tournament that the athlete plans to participate in.

[0115] If it is determined that an entry information request has been received, the distribution server device 3 executes an entry information transmission process in step S310. The entry information transmission process is a process of transmitting necessary information to other information processing devices according to the user who made the request. Note that, from the viewpoint of protecting personal information, the recipients of the entry information may be restricted. In this case, even if a transmission request is received from a non-recipient user, the entry information is not transmitted.

[0116] After receiving the entry information request, or after determining that the entry information request has not been received, the distribution server device 3 returns to the process of step S301. The distribution server device 3 repeatedly executes the processes of steps S301, S303, S305, S307, and S309 shown in FIG. 8 to determine whether various requests have been received from other information processing devices, and executes the appropriate corresponding processes.

[0117] In addition to the above, a request to change player information or a request to withdraw from membership may be monitored.

[0118] <5. Live Streaming> The distribution server device 3 may have a function for live distribution (streaming distribution) on the day of the tournament. In that case, an example of the process executed by the distribution server device 3 for live distribution will be described with reference to FIG.

[0119] In step S601, the distribution server device 3 determines whether the user who has requested live distribution is logged in, and in step S602, performs appropriate login processing. These processing steps are similar to the processing steps S401 and S402 in FIG.

[0120] After the requesting user is logged in, the distribution server device 3 receives player selection information in step S603. The player selection information is information for selecting a player who wishes to be live-distributed from among the players currently competing, and is designated by the requesting user, for example.

[0121] After receiving the player selection information, the distribution server device 3 performs a synthesis process on the captured image data in step S604, thereby generating synthetic image data such as those shown in Figs.

[0122] In step S605, the combined image data is distributed to each user by the distribution server device 3. The terminals to which the image data is distributed are, for example, the team terminal 4 or personal terminal 5 used by the requesting user.

[0123] In step S606, the distribution server device 3 determines whether or not to end the live distribution. The live distribution ends, for example, when the competition ends. Alternatively, the live distribution ends when the user performs an operation to end the distribution.

[0124] If the live distribution is not to be ended, the distribution server device 3 executes the processes of steps S604 and S605 again. The processes in steps S604 and S605 are executed for each frame of the captured image data, for example.

[0125] On the other hand, if it is determined that the live distribution should be ended, the distribution server device 3 executes a process of ending the live distribution in step S607.

[0126] For live streaming, it is preferable to use lines such as 5G (5th Generation), which offer higher speeds, larger capacity, lower latency, and more connectivity than lines such as 4G (4th Generation). This allows for a more comfortable viewing experience. Also, it may be preferable to perform live distribution to users who are in the competition venue. For example, during a competition, the top athletes are often displayed on large monitors installed in the venue, and people related to the other athletes cannot see the images they want to see. Therefore, it is possible to watch the competition of the athletes you want to see with the naked eye, but it may be difficult to see because the athletes are far away, or because of the effects of water splashes in swimming races. Therefore, it is possible to perform live distribution to users who are in the competition venue. When live streaming is limited to users within a venue, it is preferable to use a local 5G (private 5G) line or a local 4G / LTE (private 4G / LTE) line established within a limited area such as a venue. This allows for high-speed and secure streaming without using a telecommunications carrier's network.

[0127] For live distribution, different captured image data and composite image data are sent for each player, but the distribution target may be limited. For example, composite image data for a certain player may be configured to be distributed only to his / her family. In that case, after receiving the selection information of the player to be distributed in step S603, a process of determining whether the currently logged-in user is appropriate as the distribution target may be executed, and only when it is determined that the user is appropriate, each process from step S604 onward may be executed.

[0128] <6. Variations> <6-1. Modification 1 of the Editing Example of Captured Image Data> An example of editing captured image data has been described with reference to FIGS. 4 and 5, but here, another example of editing will be described with reference to FIG.

[0129] 12 shows an image of a close-up shot of a runner with an image G1 superimposed thereon to indicate a target record or a reference record. Image G1 is a line indicating the target record or the reference record, and is displayed in front of or behind the subject runner's direction of travel. Alternatively, image G1 may be a line indicating the position of the runner who is competing in the top position in the race.

[0130] In the example shown in Figure 12, a line is displayed in front of the athlete as image G1 indicating the athlete's target record, and a user who views the distributed content can understand that the athlete is lagging behind the target record by checking the positional relationship between image G1 and the athlete.

[0131] In order to superimpose the image G1, it is first necessary to calculate the superimposition position of the image G1. Specifically, the superimposition position of the image G1 is calculated from the lap time or goal time of the target record or reference record, the current position of the subject athlete, and the elapsed time from the start. As a result, for example, the superimposition position is calculated to be a position 1 m ahead of the subject athlete.

[0132] Next, it is determined where in the image the position 1 m forward is located. In order to calculate the superimposition position of the image, for example, information obtained from the imaging device 7 that captured the captured image may be used, or image analysis may be performed.

[0133] In this manner, the superimposition position of image G1 is calculated. The image generated by superimposing the image G1 is provided to the user as image content. For example, in the distribution content transmission process of Fig. 9, the user may be allowed to select distribution content on which the image G1 is superimposed as in this example. Also, in the coaching information transmission process of Fig. 10, the user may be allowed to select image content on which the image G1 is superimposed as in this example, as coaching information. Alternatively, in the live distribution process of FIG. 11, content on which the image G1 is superimposed may be live distributed.

[0134] In addition, in Fig. 12, an example in which the image G1 is superimposed on an image of a close-up of a player has been described, but the image G1 may be superimposed on an image of the entire pool displayed in the first area AR1 in Fig. 4 or the first area AR1 in Fig. 5. Of course, the image G1 may be superimposed on both the image of the entire pool displayed in the first area AR1 and the close-up image of the player displayed in the second area AR2. In addition, first, the image G1 is superimposed on the close-up image of the player displayed in the second area AR2, but when the current position of the player and the position based on the reference record or the target record are separated from each other and it becomes impossible to superimpose the image G1 on the close-up image displayed in the second area AR2, the image G1 may be switched to be superimposed on the entire image displayed in the first area AR1.

[0135] Instead of superimposing the image G1 showing the target record or the reference record, multiple images may be displayed. For example, images may be synthesized so that image data of an athlete's swim when he achieved his personal best is displayed side by side with image data of the same athlete's current race. This makes it possible to find differences in breathing timing and hand stroke timing. Of course, images may be synthesized so that image data of other swimmers are displayed side by side. For example, image data of a swimmer's swimming motion may be synthesized with image data of a world-class swimmer's swimming motion to obtain coaching information about the swimmer.

[0136] <6-2. Modification 2 of the Editing Example of Captured Image Data> In each of the above-mentioned examples, the image content shown in FIG. 4 or FIG. 5 is generated using captured image data acquired from the imaging device 7 installed in the venue. In this modification, an example will be described in which image content is generated using captured image data captured by a user's personal imaging device.

[0137] FIG. 13 shows an example of a process executed by the distribution server device 3. In step S701, the distribution server device 3 determines whether or not a distribution content generation request has been received. This request is for generating distribution content using captured image data taken by a personal user.

[0138] If the request has not been received, the distribution server device 3 executes the process of step S701 again. On the other hand, if it is determined that such a request for generating distribution content has been received, the distribution server device 3 acquires captured image data captured by the user in step S702. As a result, the information processing device such as the personal terminal 5 that sent the request performs a process of uploading the captured image data to the distribution server device 3. Note that the acquisition of the captured image data may be performed together with the reception of the request in step S701.

[0139] Next, in step S703, the distribution server device 3 acquires information for identifying the target race. This information is transmitted from the personal terminal 5 to the distribution server device 3, for example, when the user performs an operation to select the target race. Note that this information may be analyzed from metadata included in the uploaded captured image data instead of being received from the distribution server device 3. Specifically, the target tournament or race may be identified from the shooting date and time or GPS (Global Positioning System) information.

[0140] In step S704, the distribution server device 3 acquires an overall image of the target race. The overall image is captured image data of the target race identified in step S703 from among captured image data acquired from the imaging device 7 installed in the venue, and is, for example, an image of the entire pool.

[0141] The distribution server device 3 performs a synchronization process in step S705. If each captured image data used to generate the distribution content was acquired from an imaging device 7 installed in the venue, synchronization is possible by, for example, matching the time information of each imaging device 7. However, in this example, at least a portion of the captured image data was captured by an individual user, and the imaging devices are not synchronized with each other. Therefore, the synchronization process in step S705 is required.

[0142] In the synchronization process, image analysis of each captured image data can be performed to extract and synchronize timing that appears to be the same scene from the subject's characteristic movements, splashes of water, or the movements of spectators or poolside staff. Alternatively, when audio data is included in each captured image data, the audio data may be analyzed to achieve synchronization.

[0143] After synchronizing the captured image data, the distribution server device 3 acquires text information in step S706. This process is the same as the process in step S104 in FIG.

[0144] The distribution server device 3 performs a synthesis process in step S707. The synthesis process is performed based on the result of the synchronization process performed in step S705. That is, for example, the synthesis process is performed so that the captured image data displayed in the first area AR1 and the captured image data displayed in the second area AR2 shown in Fig. 4 and Fig. 5 are displayed in synchronization with each other.

[0145] By executing the series of processes shown in FIG. 13, the distribution server device 3 can generate one distribution content by synchronizing privately captured image data with officially captured image data.

[0146] <7. Summary> As explained using various examples, the distribution server device 3 is equipped with a content generation unit (content management unit 14) that generates display content (distribution content) by combining a first image captured by a first imaging device (imaging device 7) and a second image captured by the first imaging device (imaging device 7) or another imaging device (which may be a personal terminal 5, etc.) and having a part of the subject of the first image as the subject. When photographing a plurality of subjects, there is a demand for simultaneously recording and viewing images of each subject or images with different angles of view. For example, in the case of images of a swimming race, images of each subject would be close-up images of an athlete competing in lane 1 and an athlete competing in lane 8. Furthermore, images with different angles of view, for example, in the case of a race image, include an angle of view that captures the entire race and an angle of view that captures only one runner participating in the race. However, due to the shooting environment and the shortage of photographers, it is difficult for individual users to keep a variety of such images as recorded images. According to the above-mentioned configurations, display content is generated by combining official race images of the entire race with images of individual racers taken by individuals, eliminating the need for individuals to prepare both equipment for photographing the entire race and equipment for photographing the target racers. In addition, if official images are available for the images of the target racers, there is no need for individuals to take photos, improving convenience. In addition, if the official images of the entire race are of a sufficiently high resolution, close-up images of the athletes can be generated by cutting them out from the overall image, which can help minimize the amount of filming equipment required and contribute to cost reduction. In addition, by combining an overall image with images of individual players, it is possible to grasp the characteristics and areas for improvement of individual players, which can be useful in extracting coaching information.

[0147] The distribution server device 3 may include a transmission processing unit (distribution control unit 19) that transmits the display content. This makes it possible to transmit display content (eg, the distribution content shown in FIG. 4 or FIG. 5) that combines the first image and the second image to a user terminal or the like.

[0148] The distribution server device 3 may include a synchronization processing unit 17 that performs a synchronization process to synchronize the first image and the second image, as described in the modified example 2 of the editing example of the captured image data. This makes it possible to grasp the temporal relationship between the first and second images even if they are taken at different angles of view. For example, in images of a race, the overall image and images of each individual runner are synchronized, making it easier to grasp the position of each runner within the entire race.

[0149] As explained in the editing examples of captured image data (FIGS. 4 and 5), the display content (distribution content) may include superimposition information. The superimposition information is, for example, character (text) information or image information. By generating display content by superimposing text information and image information on the combination of the first and second images, the amount of information contained in the display content can be increased, allowing for a deeper understanding of the content.

[0150] As explained in the example of editing captured image data (FIG. 5), the superimposition information may be explanatory information about the first image or the second image. For example, in the case of a race image, the explanatory information includes the date and time of the race, the location of the race, the name of the race, and the names of the participants. By displaying these pieces of information in a superimposed manner, it becomes possible to easily grasp the contents of the shot image even if similar display content is available.

[0151] As explained in the modified example 1 of the editing example of the captured image data, the superimposition information may be used as comparison information for the subject of the second image. The comparison information is, for example, information (for example, image G1) showing the difference between the target record of the athlete who is the subject of the second image. Specifically, it is information showing how far ahead or behind the target record the athlete is. By overlaying this information, it is possible to understand things like the athletes' pace distribution, which would not be possible to see at the venue.

[0152] As explained in the modified example 1 of the editing example of the captured image data, the comparison information may be information indicating the performance of the subject of the second image. The performance of the subject of the second image is, for example, the past record of the athlete who is the subject of the second image. That is, it is possible to compare the current record with the past personal best record through the image. Also, for example, by superimposing an image such as a line showing the personal best record, it is possible to visually grasp the superior and inferior parts during the race, which can be utilized for future training, etc.

[0153] As explained in the first modified example of the editing example of the captured image data, the comparison information may be information indicating the results of a person other than the subject of the second image (for example, a top swimmer in the world). The results of a person other than the subject of the second image may be, for example, the record of the best performer among other runners competing in the same race, or may be a Japanese record, a world record, or the like. This allows a comparison of the athlete's target results with their results in the current race. In particular, as described above, by displaying a line based on the target record, the athlete's weak points can be visually grasped.

[0154] As explained in the coaching information extraction process, an analysis processing unit 16 is provided that performs image analysis on the subject of the second image to extract features that affect the performance of the subject of the second image, and the superimposed information (e.g., text information to be displayed in the third area AR3) may be information related to the features. The feature amount that influences the performance of the subject of the second image is, for example, in swimming, the number and frequency of breaths, or the number of strokes of the hands or the number of breaths. By extracting and superimposing features that affect performance, it is possible to analyze the player's movements in more detail and obtain coaching information to improve performance.

[0155] As explained in each example of the distribution content transmission process (Figure 9), an authentication processing unit 18 is provided that performs user authentication based on received user information, and a transmission processing unit (distribution control unit 19) may select and transmit display content (distribution content) based on the authentication result. Display content is image content that includes personal information. Such content is sent only if user authentication is successful. This makes it possible to protect personal information.

[0156] As explained in the distribution content transmission process (FIG. 9), a content management unit 14 may be provided that sets the disclosure range for each display content (distribution content). There are cases where viewing of the display content needs to be restricted depending on the subject athlete, and cases where it is not. Since the scope of disclosure can be set, content management can be carried out in accordance with the user's intentions.

[0157] As described in the distribution content transmission process (Figure 9), an authentication processing unit 18 may be provided that performs user authentication based on the received user information, and a transmission processing unit (distribution control unit 19) may transmit display content (distribution content) based on the authentication result and the disclosure range. For example, the user is identified by user authentication, and a decision is made as to whether or not to accept the request of the user based on the disclosure range set in the display content. This makes it possible to manage content in line with the user's intentions and to distribute the content appropriately.

[0158] As explained in variant example 2 of the editing example of captured image data (Figure 13), the synchronization processing performed by the synchronization processing unit 17 may involve image comparison between a first image (e.g., an image to be displayed in the first area AR1) and a second image (e.g., an image to be displayed in the second area AR2). For example, when the first and second images are moving images, the synchronization process is performed by comparing parts of both images and calculating the degree of similarity. This makes it possible to synchronize both images even if timestamp information is not linked to the images. Also, even if timestamp information is linked to the images, it is difficult to synchronize both images based on the timestamp information if they are captured by unsynchronized imaging devices. Even in such cases, this configuration makes it possible to search for matching points by image processing and synchronize the images.

[0159] As described in the live distribution (FIG. 11), the transmission processing unit (distribution control unit 19) may perform streaming distribution. For example, during a competition, a first official image capturing the entire event from a bird's-eye view and a second official image focusing on a specific athlete will be combined and streamed. Furthermore, by making the second image different for each player, different display content is streamed for each player, which makes it possible to stream display content appropriate for each user in a situation where different users desire different images.

[0160] As mentioned above, 5G / local 5G (or local 4G / LTE) lines can be used for live streaming. In this case, it is also possible to use network slicing technology, which virtualizes the network and divides (slices) network resources, and uses different slices depending on the purpose and priority of the information being communicated. Specifically, for example, a distribution server device 3 distributes various data and image data to each terminal to which the data is to be distributed via a base station that is installed in the venue and that creates a local 5G communication environment in the venue and controls the communication. In this case, for example, each terminal may perform a process of synthesizing image data or superimposing various data on image data. At this time, the distribution server device 3 generates information for identifying the slice to be assigned depending on the purpose and characteristics of the data to be distributed, and transmits the data and information (metadata) for identifying the slice to which the signal should be assigned to the base station in association with each other. The base station assigns each piece of received data to a slice specified by the corresponding metadata, and transmits each piece of data to each terminal to which the data is to be distributed using the assigned slice. As an example of slice allocation, image data that is preferably viewable without delay may be allocated to a slice for low-latency transmission. In a case where image data delays can be tolerated, such as when the main focus is on watching the race visually, image data may be allocated to a slice for large-volume transmission, and various information that is preferably known in a timely manner and is superimposed on the image data (e.g., progress information such as lap times) may be allocated to a slice for low-latency transmission.

[0161] Although the above example uses competitive swimming as an example, other sports may be the target. For example, it may be track and field events such as the 100m sprint or 110m hurdles, or team sports such as synchronized swimming. In addition to sports, it may also be applied to brass band concerts and orchestra concerts. Specifically, in the case of a team sport, image data of the entire team may be used as an overall image, and image data of a close-up of one athlete may be used as a close-up image, and the distribution content may be generated by combining these. Alternatively, in the case of a brass band concert, image data of all the performers can be used as an overall image, and image data of a close-up of one of the performers can be used as a close-up image, and these can be combined to generate the distribution content. In other words, the various configurations described above can be applied to performances or competitions involving multiple people.

[0162] The program of the embodiment is a program that causes, for example, a CPU, a DSP, or a device including these to execute each process shown in each of Figures 6, 7, 8, 9, 10, 11, and 13. That is, the program of the embodiment causes the control unit of the distribution server device 3 to execute a generation process for generating content for display by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of the subject of the first image as the subject. The above-mentioned distribution server device 3 can be realized by such a program.

[0163] A program for implementing such a distribution server device 3 can be recorded in advance in a HDD as a recording medium built into a device such as a computer device, or in a ROM in a microcomputer having a CPU. Alternatively, the software may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as a so-called package software. Furthermore, such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0164] Furthermore, such a program is suitable for a wide range of applications of the distribution server device 3 of the embodiment. For example, by downloading the program to a device equipped with a camera function, such as a mobile terminal device such as a smartphone or tablet, a mobile phone, a personal computer, a game device, a video device, or a PDA (Personal Digital Assistant), the smartphone or the like can function as the distribution server device 3 of the present disclosure.

[0165] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0166] <8. This Technology> (1) The content generating unit generates a display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of a subject of the first image as a subject. Information processing device. (2) a transmission processing unit for transmitting the display content; The information processing device according to (1) above. (3) A synchronization processing unit that performs a synchronization process to synchronize the first image and the second image. An information processing device according to any one of (1) to (2) above. (4) The display content includes overlay information. An information processing device according to any one of (1) to (3) above. (5) The overlay information is explanatory information about the first image or the second image. The information processing device according to (4) above. (6) The superimposition information is comparison information for the subject of the second image. An information processing device according to any one of (4) to (5) above. (7) The comparison information is information indicating a result for the subject of the second image. The information processing device according to (6) above. (8) The comparative information is information indicating the performance of a person other than the subject of the second image. An information processing device according to any one of (6) to (7) above. (9) an analysis processing unit that performs image analysis on a subject of the second image and extracts a feature amount that affects a grade of the subject of the second image; The superimposition information is information regarding the feature amount. An information processing device according to any one of (4) to (8) above. (10) an authentication processing unit that performs user authentication based on the received user information; The transmission processing unit selects and transmits the display content based on a result of the authentication. The information processing device according to (2) above. (11) Equipped with a content management section that sets the disclosure scope for each display content The information processing device according to (2) above. (12) an authentication processing unit that performs user authentication based on the received user information; The transmission processing unit transmits the display content based on the authentication result and the disclosure range. The information processing device according to (2) above. (13) In the synchronization process, an image comparison is performed between the first image and the second image. The information processing device according to (3) above. (14) The transmission processing unit performs streaming distribution. The information processing device according to (2), (10), (11) or (12). (15) A first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of the subject of the first image as a subject are combined to generate display content. An information processing method executed by an information processing device. (16) A generation process for generating display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of a subject of the first image as a subject. A program to be executed by an information processing device. [Explanation of symbols]

[0167] 3. Distribution server device 7. Imaging device 14 Content Management Department 16 Analysis Processing Section 17 Synchronization processing section 18 Authentication processing section 19 Distribution control section (transmission processing section)

Claims

1. a content generating unit that generates a display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of a subject of the first image as a subject; an analysis processing unit that performs image analysis on a subject of the second image and extracts a feature amount that affects a grade of the subject of the second image; the display content includes overlay information; The superimposition information is information regarding the feature amount. Information processing device.

2. a transmission processing unit for transmitting the display content; The information processing device according to claim 1 .

3. A synchronization processing unit that performs a synchronization process to synchronize the first image and the second image. The information processing device according to claim 1 .

4. The overlay information is explanatory information about the first image or the second image. The information processing device according to claim 1 .

5. The superimposition information is comparison information for the subject of the second image. The information processing device according to claim 1 .

6. The comparison information is information indicating a result for the subject of the second image. The information processing device according to claim 5 .

7. The comparative information is information indicating the performance of a person other than the subject of the second image. The information processing device according to claim 5 .

8. an authentication processing unit that performs user authentication based on the received user information; The transmission processing unit selects and transmits the display content based on a result of the authentication. The information processing device according to claim 2 .

9. Equipped with a content management section that sets the disclosure scope for each display content The information processing device according to claim 2 .

10. an authentication processing unit that performs user authentication based on the received user information; The transmission processing unit transmits the display content based on the authentication result and the disclosure range. The information processing device according to claim 9.

11. In the synchronization process, an image comparison is performed between the first image and the second image. The information processing device according to claim 3 .

12. The transmission processing unit performs streaming distribution. The information processing device according to claim 2 .

13. generating a display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of a subject of the first image as a subject; performing an image analysis on the subject of the second image to extract a feature amount that affects a grade of the subject of the second image; the display content includes overlay information; The superimposition information is information regarding the feature amount. An information processing method executed by an information processing device.

14. a generation process for generating display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device and having a part of a subject of the first image as a subject; and performing an image analysis on the subject of the second image to extract a feature amount that affects the performance of the subject of the second image, the display content includes overlay information; The superimposition information is information regarding the feature amount. A program to be executed by an information processing device.

Images