Information processing device, information processing method, and program
By combining images captured by multiple imaging devices, synchronous and detailed display content is generated, the problem of difficulty in seeing the entire competition and player movements at the same time in the competition is solved, improving user understanding and providing guidance information, and reducing device requirements and costs .
Patent Information
- Application Number
- CN202080086211.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-12-20
- Filing Date
- 2020-11-13
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-11-13
AI Technical Summary
In a competition, it is difficult to see the relationship between the movements of the entire competition and the movements of each contestant at the same time, and prior art is difficult to effectively combine streaming data captured by multiple television cameras to generate appropriate content as a whole.
Through the information processing device, the first image captured by the first imaging device and the second image captured by the second imaging device are combined to generate display content, including synchronization processing and superimposed information, such as score comparison and feature quantity analysis, to ensure that the user can understand the competition Condition.
It realizes overall mastery of the subject's movements in the competition, provides detailed competition information and guidance, improves user experience, reduces the demand for a variety of imaging devices, and reduces costs.
Smart Images

Figure CN114793486B_ABST
Abstract
Description
Technical Field
[0001] The present technology relates to an information processing device, an information processing method, and a program, and particularly to a technology for generating content in which two images are combined. Background Art
[0002] For example, when watching a sports game, it is difficult to visually recognize the performances of multiple subjects at the same time. On the other hand, Patent Document 1 discloses combining streaming data captured by multiple television cameras to generate one combined streaming data, and distributing the combined streaming data via a network in response to a request from a viewer terminal.
[0003] Citation List
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2003-18583 Summary of the invention
[0006] Problems to be solved by the present invention
[0007] However, for example, in a competition such as a match where comparison with others is important, there is a need to see the movement of the entire competition and the relationship between the movements of each player at the same time.
[0008] Therefore, an object of the present technology is to generate appropriate content for grasping the motion of a subject as a whole.
[0009] Solution to the problem
[0010] An information processing device according to the present technology includes: a content generation unit, which generates display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, wherein the second image takes a part of a subject in the first image as a subject.
[0011] In the case of imaging a plurality of subjects, there is a need to simultaneously record an image of each subject or an image with a changed viewing angle, or a need for browsing.
[0012] The above-mentioned information processing apparatus may include a synchronization processing unit that performs synchronization processing to synchronize the first image and the second image.
[0013] Thus, display content obtained by combining the first image and the second image can be transmitted to a user terminal or the like.
[0014] The above-mentioned information processing apparatus may include a synchronization processing unit that synchronizes the first image and the second image.
[0015] Thus, even when the first image and the second image are captured at different angles of view, the temporal relationship can be grasped.
[0016] In the above-mentioned information processing apparatus, the display content may include superimposed information.
[0017] The superimposed information is, for example, character (text) information, image information, and the like.
[0018] In the above-mentioned information processing apparatus, the superimposed information may be explanation information about the first image or the second image.
[0019] For example, in the case of a competition image, the description information is description information that facilitates understanding when viewing the image, such as the date and time of the event, the venue, the name of the competition, and the names of the contestants.
[0020] In the above information processing apparatus, the superimposition information may be comparison information on the subject of the second image.
[0021] The comparison information is, for example, information indicating a difference from a target record of a player who is a subject of the second image. Specifically, the comparison information is information indicating how much the subject is ahead of or behind the target record.
[0022] In the above-mentioned information processing apparatus, the comparison information may be information indicating a performance of a subject of the second image.
[0023] The performance of the subject of the second image is, for example, the past record of the player who is the subject of the second image.
[0024] In the above-mentioned information processing apparatus, the comparison information may be information indicating a performance of a person other than the subject of the second image.
[0025] The results of the people other than the subject of the second image are, for example, the records of the best players among other players participating in the same competition. Alternatively, the results may be results such as Japanese records or world records.
[0026] The information processing device may include an analysis processing unit that performs image analysis on the subject of the second image and extracts a feature quantity that affects the performance of the subject of the second image. The superimposition information may be information related to the feature quantity.
[0027] For example, in the case of swimming, the feature quantity that affects the performance of the subject of the second image is the number or frequency of breathing, the number of hand strokes, and the like.
[0028] The information processing device may include an authentication processing unit, which performs user authentication based on the received user information. The transmission processing unit may select and transmit display content based on the authentication result.
[0029] Display content is image content including personal information. Such content is only sent if the user authentication is successful.
[0030] The above-mentioned information processing apparatus may include a content management unit that sets a disclosure range for each display content.
[0031] Depending on the subject player, the displayed content may or may not have viewing restrictions.
[0032] The information processing device may include an authentication processing unit configured to perform user authentication based on the received user information. The transmission processing unit may transmit the display content based on the authentication result and the disclosure range.
[0033] For example, the user is specified by user authentication, and whether to accept the user's request is determined based on the disclosure range set in the display content.
[0034] In the above information processing apparatus, in the synchronization process, image comparison between the first image and the second image may be performed.
[0035] For example, when the first image and the second image are moving images, the synchronization process is performed by partially comparing the two images with each other to calculate the degree of coincidence.
[0036] In the above-mentioned information processing device, the sending processing unit can perform streaming distribution.
[0037] For example, streaming distribution is performed by combining an official first image obtained by capturing an overall image in a bird's-eye view during a competition and an official second image focusing on a specific player.
[0038] The information processing method of the present technology is performed by an information processing device. The method includes: generating display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, the second image having a part of a subject in the first image as a subject.
[0039] The program of the present technology causes the information processing device to perform: generation processing, which generates display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, wherein the second image has a part of the subject in the first image as a subject.
[0040] Therefore, appropriate content for grasping the movement of the subject as a whole can be generated. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1is a diagram illustrating the configuration of a distribution system according to the embodiment.
[0042] Figure 2 is a block diagram of an information processing apparatus according to an embodiment.
[0043] Figure 3 is a functional block diagram of a distribution server device in the embodiment.
[0044] Figure 4 is an example of distributing content according to an embodiment.
[0045] Figure 5 is another example of distributing content according to an embodiment.
[0046] Figure 6 is a flowchart illustrating an example of distribution content generation processing according to the embodiment.
[0047] Figure 7 is a flowchart illustrating an example of guidance information extraction processing according to the embodiment.
[0048] Figure 8 is a flowchart illustrating an example of request response processing according to the embodiment.
[0049] Fig. 9 is a flowchart illustrating an example of distribution content sending processing according to the embodiment.
[0050] Fig.10 is a flowchart illustrating an example of guidance information transmission processing according to the embodiment.
[0051] Fig.11 is a flowchart illustrating an example of live broadcast distribution processing according to an embodiment.
[0052] Fig.12 : is a diagram for explaining a first modification example of the editing example of captured image data according to the embodiment.
[0053] Fig.13 is a flowchart illustrating an example of distribution content generation request reception processing according to the embodiment. DETAILED DESCRIPTION
[0054] Hereinafter, the embodiments will be described in the following order.
[0055] <1. System Configuration>
[0056] <2. Distribution Server Equipment Configuration>
[0057] <3. Editing Example of Captured Image Data>
[0058] <4. Processing Example in Distribution Server Device>
[0059] <4-1. Distribution content generation process>
[0060] <4-2. Guidance Information Extraction Process>
[0061] <4-3. Request response processing>
[0062] <5. Live broadcast distribution>
[0063] <6. Modifications>
[0064] <6-1. First Modification of Editing Example of Captured Image Data>
[0065] <6-2. Second Modification of the Editing Example of Captured Image Data>
[0066] <7. Conclusion>
[0067] <8. This technology>
[0068] <1. System Configuration>
[0069] Reference Figure 1 , the configuration of the distribution system 1 in this embodiment is described. Note that in the following description, a system for distributing images related to swimming as a sports competition will be described as an example of the distribution system 1. The "image" here includes both moving images and still images.
[0070] The distribution system 1 includes a distribution server device 3 , a team terminal 4 , and a personal terminal 5 which are communicatively connected to each other via a communication network 2 .
[0071] Furthermore, the distribution server device 3 can communicate with the venue facility system 6. The venue facility system 6 includes a plurality of imaging devices 7.
[0072] As the communication network 2 , for example, the Internet, a home network, a local area network (LAN), a satellite communication network, and other various networks are conceived.
[0073] The distribution server device 3 is, for example, an information processing apparatus managed by an operation organization that operates various championships and events related to swimming, and includes a personal computer (PC), a workstation, and the like.
[0074] The distribution server device 3 manages the schedule of swimming championships, swimming events, and competitors (players). The distribution server device 3 sends schedule information and information about competitions and competitors to the team terminal 4 and the personal terminal 5 .
[0075] Furthermore, the distribution server device 3 receives an entry request for the tournament, and executes an entry process for the tournament.
[0076] The distribution server device 3 is a device that distributes images captured by an imaging device 7 installed in a swimming venue to the team terminal 4 and the personal terminal 5. The distribution server device 3 can communicate with the imaging device 7 included in the venue facility system 6.
[0077] Note that communication between the distribution server device 3 and the imaging device 7 may be achieved via a network or may be achieved by direct connection. In addition, the venue facility system 6 may include a management terminal that comprehensively manages each imaging device 7, and the distribution server device 3 may communicate with the imaging device 7 via the management terminal.
[0078] Furthermore, the venue facility system 6 may be connected to the communication network 2 , and the distribution server device 3 may communicate with the imaging device 7 via the communication network 2 .
[0079] The distribution server device 3 generates content for distribution (hereinafter referred to as "distribution content") by using the captured image data obtained from the imaging device 7. The distribution content is, for example, image data obtained by combining the captured image data obtained from a plurality of imaging devices 7, and text information or image data such as explanatory text information and metadata may be superimposed.
[0080] The distributed content is content to be displayed on each information processing device, and thus may also be referred to as display content.
[0081] The distribution content generated by the distribution server device 3 is appropriately distributed to the team terminal 4 and the personal terminal 5 .
[0082] The distribution server device 3 may be an information processing device temporarily installed in the venue during the tournament. In addition, in this case, the distribution server device 3 may be configured by cooperation between a permanent information processing device, which is an information processing device owned by the operating organization and performs various processes that are frequently performed except during the tournament, such as schedule management, player information management, and registration processing, and a temporarily installed information processing device, which is an information processing device temporarily installed in the venue and performs various processes that are particularly performed during the tournament, such as control of the imaging device 7 and processing for captured image data.
[0083] In the following example, an example in which the distribution server device 3 includes both a permanent information processing apparatus and a temporarily installed information processing apparatus will be described.
[0084] The team terminal 4 is an information processing device owned by a swimming team or a club team, such as a PC, a personal digital assistant (PDA), etc. For example, the team terminal 4 can only be used by users (staff or competitors) belonging to the swimming team, and can make various requests to the distribution server device 3. For example, the various requests include a request to send distribution content, a request to send tournament information, etc.
[0085] Furthermore, the team terminal 4 may send a registration request for the tournament to the distribution server device 3 by specifying the tournament, competition items, and competitors.
[0086] The team terminal 4 manages information related to the competitors of the swimming team. For example, the team terminal 4 can respond to a request from the individual terminal 5 and send the scores, competition information, etc. of the competitors.
[0087] The personal terminal 5 is an information processing device owned by the contestant, for example, a device having a communication function, such as a portable terminal device such as a smartphone or a tablet, a mobile phone, a PC, a game device, a video device, or a PDA.
[0088] The personal terminal 5 may send a request for obtaining the tournament results and personal information to the team terminal 4 to obtain various information. Furthermore, the personal terminal 5 may be configured to be able to receive the tournament information and distribution content from the distribution server device 3 .
[0089] Furthermore, the personal terminal 5 can send a login request to the team terminal 4 or the distribution server device 3. For example, by sending an identification (ID) and a password as login information to each device so as to acquire various information, an authentication process is performed in each device.
[0090] The venue facility system 6 includes the imaging device 7 installed in the swimming venue. As described above, a management terminal for overall management of each imaging device 7 may be included.
[0091] The imaging device 7 is a camera installed in a swimming pool, etc., and is set up as, for example, an imaging device 7 for imaging the entire swimming pool, an imaging device 7 for capturing an image of each lane, an imaging device 7 for imaging an electronic bulletin board, an imaging device 7 for imaging an auditorium, etc.
[0092] <2. Distribution Server Equipment Configuration>
[0093] The configuration of the distribution server device 3 will be described. As described above, here, an example will be described in which the distribution server device 3 has a management function of the imaging device 7 installed in the swimming pool. For example, the distribution server device 3 includes a plurality of information processing apparatuses.
[0094] Each of the plurality of information processing devices has, for example, Figure 2 The composition shown in the diagram.
[0095] The CPU 71 of the information processing apparatus executes various processes in accordance with the program stored in the ROM 72 or the program loaded from the storage unit 79 to the RAM 73. The RAM 73 also appropriately stores data and the like necessary for the CPU 71 to execute various processes.
[0096] The CPU 71, the ROM 72, and the RAM 73 are connected to each other via a bus 74. To the bus 74, an input / output interface 75 is also connected.
[0097] An input unit 76 including an operator and an operating device is connected to the input / output interface 75 .
[0098] For example, as the input unit 76 , various operators and operating devices such as a keyboard, a mouse, a key, a knob, a touch panel, a touch pad, and a remote controller are conceivable.
[0099] The input unit 76 detects an operation by a user, and the CPU 71 interprets a signal corresponding to the input operation.
[0100] In addition, a display unit 77 including an LCD, an organic EL panel, or the like, and an audio output unit 78 including a speaker or the like are connected to the input / output interface 75 integrally or separately.
[0101] The display unit 77 is a display unit that performs various displays, and includes, for example, a display device provided in a casing of the distribution server device 3 , an independent display device connected to the distribution server device 3 , and the like.
[0102] The display unit 77 performs display of images for various image processing, moving images to be processed, etc. on the display screen based on the instructions from the CPU 71. In addition, the display unit 77 displays various operation menus, icons, messages, etc. based on the instructions from the CPU 71, that is, displays as a graphical user interface (GUI).
[0103] In some cases, a storage unit 79 including a hard disk, a solid-state memory, or the like, and a communication unit 80 including a modem or the like are connected to the input / output interface 75 .
[0104] The communication unit 80 performs communication processing via a transmission path such as the Internet, wired / wireless communication with various devices, bus communication, and the like.
[0105] In the case of the present embodiment, the communication unit 80 has a function of communicating with the imaging device 7 .
[0106] A drive 82 is also connected to the input / output interface 75 as necessary, and appropriately mounts a removable recording medium 81 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0107] Various computer programs and the like can be read from the removable recording medium 81 via the drive 82. The computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as necessary.
[0108] In the distribution server device 3, for example, software for the processing of the present disclosure may be installed via network communication through the communication unit 80, or via the removable recording medium 81. Alternatively, the software may be pre-installed in the ROM 72, the storage unit 79, or the like.
[0109] For example, by such software (application), the following is constructed in the CPU 71 of the distribution server device 3: Figure 3 The functional composition is shown in the diagram.
[0110] The distribution server device 3 has various functions such as a camera controller 11, a switcher 12, a player information management unit 13, a content management unit 14, an image editing processing unit 15, an analysis processing unit 16, a synchronization processing unit 17, an authentication processing unit 18, and a distribution control unit 19.
[0111] The camera controller 11 controls the imaging device 7 installed in the venue. Specifically, the control includes control for causing the imaging device 7 to perform a rotation operation in the pan direction or the pitch direction, control for performing a push-in operation or a zoom-out operation, control for changing the quality of captured image data, such as image quality or color tone, etc. In addition, the camera controller 11 can also perform control for transmitting the captured image data captured by the imaging device 7, etc.
[0112] The switcher 12 performs processing of changing a live distribution image, processing of switching an image to be displayed on a large display installed in a venue, and the like by switching images of the plurality of imaging devices 7 .
[0113] Note that the camera controller 11 and the switch 12 may be implemented as independent devices.
[0114] The player information management unit 13 manages various information related to the players belonging to the operating organization. For example, information such as the player's name, player ID, team information, address, contact address, date of birth, and international age at the time of the tournament for identifying an individual, information related to the tournament participated in, record information, etc. are used as the player information.
[0115] The content management unit 14 manages image content information generated for distribution. Image content is, for example, content generated by combining captured image data acquired from a plurality of imaging devices 7, distribution content for distribution to the team terminal 4 or the personal terminal 5, and display content to be displayed on each terminal.
[0116] The content management unit 14 manages the image content and metadata or descriptive text information associated with the image content. The metadata or descriptive text information is, for example, the name of the subject (player name), the name of the championship, the venue of the championship, the imaging date and time, the swimming competition event, the lane number used by the player, the swimming record, the ranking, etc. In addition, information for specifying the imaging device 7 for imaging the player, the imaging settings of the imaging device 7, etc. may be recorded.
[0117] The content management unit 14 sets and stores a disclosure range for each image content. For example, the disclosure range may include "distribution only to the player himself", "distribution to the team to which the player belongs", and "distribution to anyone".
[0118] The image editing processing unit 15 performs editing processing for the captured image data acquired from the imaging device 7. Various editing processing can be considered, for example, processing including processing of combining a plurality of captured image data acquired from a plurality of imaging devices 7 to generate one image data, processing of superimposing text information or an image, processing of cutting out an image in a predetermined range from the acquired captured image data, and the like.
[0119] For example, the text information to be superimposed includes player information, tournament information, swimming records, etc. In addition, the images to be superimposed are images for intuitively indicating the past performance of the players who are the subjects of the acquired captured image data, images for intuitively indicating record information of each tournament and country, etc.
[0120] The analysis processing unit 16 is a processing unit that performs image analysis on the captured image data obtained from the imaging device 7 and obtains guidance information. For example, in the case of swimming, the number of breaths, the number of strokes, the number of kicks, etc. performed when swimming 50 meters are obtained through image analysis. In addition, the position of the player and the speed at that time (swimming speed) can be calculated through image analysis.
[0121] This information is guidance information that can be used for guidance. When extracting the guidance information, for example, a deep learning method such as a convolutional neural network (CNN) can be used.
[0122] In the case of combining a plurality of captured image data to generate one distribution content, the synchronization processing unit 17 performs a process of synchronizing the plurality of captured image data.
[0123] The authentication processing unit 18 performs authentication processing of determining whether the user is a member based on ID information and password information received from other terminals such as the team terminal 4 or the personal terminal 5 .
[0124] Note that the authentication processing unit 18 may be configured to send, for example, a two-dimensional barcode as information indicating membership to a user or terminal for which the authentication processing has succeeded. For example, a user who has received the two-dimensional barcode information can smoothly enter the venue or enter a relevant personnel area by placing the two-dimensional barcode over a reader installed in the venue. In addition, the two-dimensional barcode can also be used when obtaining imaging permission in a swimming venue.
[0125] The distribution control unit 19 acquires the image content from the database and performs distribution control to distribute the image content to the terminal that has sent a distribution request for the image content. In addition, the distribution control unit 19 may perform distribution control based on the public range of the image content.
[0126] In addition, the distribution control unit 19 can perform control for live distribution of captured image data captured in real time in a swimming pool. For example, live images can be distributed to users who come to the swimming pool as family members. At this time, different live images can be distributed to each user. Specifically, for example, the captured image data (overall image) from the imaging device 7 that images the entire swimming pool in a bird's-eye view and the captured image data (propulsion image, personal image) from the imaging device 7 that images the individual players in propulsion mode can be combined in the image editing processing unit 15, and the individual image content for each player obtained by the combination can be distributed to each user as a live image. Thus, the user can receive image content created specifically for his / her child.
[0127] Note that, among the various functions included in the above-described distribution server device 3 , at least some of the functions may be functions provided by cloud computing.
[0128] Furthermore, for example, in the case where the distribution server device 3 is not installed in a swimming pool, the distribution server device 3 may be provided as a stationary server device owned by an operation organization, and may have various functions.
[0129] <3. Editing Example of Captured Image Data>
[0130] An example of image content edited by the image editing processing unit 15 of the distribution server device 3 will be described.
[0131] There are cases where a user desires to view images in which a player related to the user is captured in a push mode during a competition, images in which the entire competition is imaged in a bird's-eye view, and information of an electronic bulletin board on which a record related to the competition is displayed. In order to meet such needs, for example, a video such as the one shown in FIG. 1 is generated by editing the captured image data acquired from the imaging device 7. Figure 4 The image content shown in the figure.
[0132] Figure 4 A scenario illustrating image content generated for distribution to users.
[0133] The image content includes a plurality of areas. For example, the first area AR1 is an area in which an image for intuitively recognizing the progress of the entire game is displayed, and captured image data acquired from the imaging device 7 that images the entire swimming pool is displayed. Hereinafter, image data generated by combining a plurality of image data in this manner will be referred to as "synthesized image data".
[0134] In the second area AR2, captured image data acquired from the imaging device 7 that captures a propulsion image of an individual player is displayed.
[0135] Note that, in the case where the image obtained by capturing the entire swimming pool has a sufficiently high resolution, an image obtained by cutting out a specific area imaging an individual player from among the image displayed in the first area AR1 and zooming in on the area may be displayed in the second area AR2. That is, an image may be displayed in each of the first area AR1 and the second area AR2 by using captured image data acquired from one imaging device 7.
[0136] Figure 4 In the example, two areas are set in which captured image data is displayed, but this is of course just an example, and three or more areas may be set. In this case, for example, Figure 4 As illustrated in FIG. 1 , in addition to the first area AR1 and the second area AR2 , a third area AR3 in which captured image data acquired from the imaging device 7 that images the electronic bulletin board is displayed may be provided.
[0137] In addition, the respective areas may be set to have the same size, or may be set to have different sizes. In addition, a certain area may be arranged to be superimposed on another area.
[0138] Note that character information may be displayed instead of an image. An example of this is shown in Figure 5 The character information is, for example, description information for describing an image.
[0139] Figure 5The diagram illustrates an example in which, in addition to the first area AR1 displaying an image captured as a whole in a bird's-eye view and the second area AR2 displaying an image obtained by imaging individual players in a push mode, a third area AR3 for displaying text information extracted from information displayed on the electronic bulletin board is provided.
[0140] The third area AR3 is an area displayed superimposed on the second area AR2.
[0141] For example, by performing image analysis on captured image data obtained by imaging the electronic bulletin board, text information displayed in the third area AR3 as explanatory information can be obtained. Character recognition is performed on the captured image data of the electronic bulletin board to obtain text information. Alternatively, text information sent for display on the electronic bulletin board can be obtained, and text information to be displayed as explanatory information can be extracted therefrom.
[0142] Furthermore, time information may be acquired from a device that acquires information of contestants participating in a competition from a database and manages process information of swimming (eg, round trip time information for each 50 m), and text information may be generated from the acquired information.
[0143] In addition, as text information, the championship name, championship venue, imaging date and time, swimming competition events, the player's name displayed in the second area AR2, the lane number used by the player, swimming records, rankings, personal best information, etc. can be obtained.
[0144] When displaying text information, you can Figure 4 The composite image data and text information shown in the diagram are respectively sent to other terminals such as the team terminal 4 and the personal terminal 5, and the text information can be superimposed and displayed on the composite image data through processing by the other terminals. In this case, the user can easily freely set the position of the superimposed text information and easily set the switching between display and non-display of the text information.
[0145] Alternatively, the text information may be transmitted to other terminals in a state where it is incorporated in the composite image data in advance as image information by the image editing processing unit 15 of the distribution server device 3 .
[0146] Furthermore, text information may be displayed without setting the third area AR3. For example, when text information is incorporated in captured image data to be displayed in the second area AR2, it is not necessary to set the third area AR3 for displaying the text information.
[0147] <4. Processing Example in Distribution Server Device>
[0148] <4-1. Distribution content generation process>
[0149] Various processes executed by the distribution server device 3 will be described.
[0150] First, refer to Figure 6 The process of generating distribution content will be described. The distribution content generating process is a process executed after a swimming championship, for example, and is a process of generating individual distribution content for each player.
[0151] In the distribution content generation process, at step S101, the distribution server device 3 selects a player. The distribution content generation process is, for example, a process performed after the end of a tournament by using captured image data acquired from a plurality of imaging devices 7. Thus, at step S101, the process is a process of selecting a player from among the players participating in the tournament. Note that when there are players participating in a plurality of games in one tournament, for example, in the case where there are players participating in both the preliminaries and the finals, a process of selecting both the game and the player is performed at step S101.
[0152] In step S102, the distribution server device 3 performs a process of selecting an entire image. Specifically, the captured image data acquired from the imaging device 7 that captures the entire image in a bird's-eye view is selected. In addition, the acquired captured image data is for a specific game in which the player selected in step S101 participates.
[0153] In step S103, the distribution server device 3 performs a process of selecting a player's personal image. This process is about the captured image data of the player specified in step S101. In the case where the player participates in a plurality of games, one captured image data is selected based on the game information selected in step S101.
[0154] In step S104, the distribution server device 3 acquires text information. This process is performed so as to acquire, for example, Figure 5 The information to be acquired is, for example, information such as swimming venue information, championship name, competition information, lane number used by the player in the competition, player name, performance information in the competition such as the time taken for one round trip, and personal best performance.
[0155] Note that, in the case where text information is not used, it is not necessary to perform the processing of step S104.
[0156] In step S105, the distribution server device 3 performs processing for combining captured image data. By this processing, a Figure 4 Or the distribution content shown in the diagram in 5.
[0157] By the distribution server device 3 executing the process of step S106 , the generated distribution content is stored in the storage unit 79 or the like.
[0158] In step S107, the distribution server device 3 determines whether the above-mentioned various types of processing have been completed for all players participating in the tournament. In the case where there are still unprocessed players, the distribution server device 3 returns to the processing of step S101.
[0159] On the other hand, if there are no unprocessed players, the distribution server device 3 ends the Figure 6 A series of processing shown in the diagram.
[0160] By executing Figure 6 The distribution content generation process shown in the diagram generates different distribution content for each player participating in the tournament.
[0161] Note that although it is stated that all players participating in the tournament Figure 6 The processing is an example of the processing shown in the diagram, but the processing may be performed for some players. In addition, when a distribution request for distributing content is received from the team terminal 4 or the personal terminal 5, it may be appropriately executed. Figure 6 In the case of the distribution content generation processing illustrated in FIG, it is not necessary to perform unnecessary processing, so that the processing burden of the distribution server device 3 is reduced.
[0162] <4-2. Guidance Information Extraction Process>
[0163] refer to Figure 7 , explaining the guidance information extraction processing performed by the distribution server device 3. The guidance information extraction processing is a processing for extracting information for guidance by analyzing the captured image data acquired from the imaging device 7. The guidance information extraction processing is a processing performed after a tournament, for example.
[0164] In step S201, the distribution server device 3 selects a player. Figure 6 A process similar to the process of step S101 in is a process of selecting a player who has participated in the tournament.
[0165] In step S202, the distribution server device 3 selects a player's personal image. Figure 6 The processing is similar to the processing of step S102 in .
[0166] In step S203, the distribution server device 3 performs image analysis processing. The image analysis processing is a process of analyzing the captured image data selected in step S202 and extracting feature amounts. In the analysis, for example, a deep learning method such as CNN can be used.
[0167] In step S204, the distribution server device 3 extracts guidance information. In this process, the number of breaths, the number of strokes, the number of kicks, etc. are extracted as guidance information based on the analysis results of the previous image analysis process. In addition, the change in swimming speed from the start to the end of the competition can be extracted.
[0168] In step S205, distribution server device 3 performs processing for storing guidance information. Specifically, guidance information is stored in association with a tournament, a match, or the like in which the player participates.
[0169] In step S206, the distribution server device 3 determines whether the above-mentioned various types of processing have been completed for all players participating in the tournament. In the case where there are still unprocessed players, the distribution server device 3 returns to the processing of step S201.
[0170] On the other hand, if there are no unprocessed players, the distribution server device 3 ends the Figure 7 A series of processing shown in the diagram.
[0171] By executing Figure 7 The guidance information extraction process shown in the diagram extracts guidance information for each player participating in the tournament.
[0172] Note that although it is stated that all players participating in the tournament Figure 7 The processing is an example of the processing shown in the diagram, but the processing can be performed for some players. In addition, when a transmission request of the guidance information is received from the team terminal 4 or the individual terminal 5, it can be appropriately executed. Figure 7 In the case of the guidance information extraction processing illustrated in FIG, it is not necessary to perform unnecessary processing, so that the processing burden of the distribution server device 3 is reduced.
[0173] Note that the guidance information extracted here can be displayed as text information in Figure 5 In the third area AR3 shown in the diagram.
[0174] <4-3. Request response processing>
[0175] The distribution server device 3 receives various requests from the team terminal 4 or the personal terminal 5. Then, the distribution server device 3 confirms the reception of the various requests, and performs response processing for each request.
[0176] Figure 8 An example of request response processing performed by the distribution server device 3 is illustrated in FIG.
[0177] In step S301, the distribution server device 3 determines whether a member registration request is received. In the case where it is determined that a member registration request is received, the distribution server device 3 performs a member registration process in step S302 as a response process.
[0178] In the member registration process, for example, web page data for member registration is sent to the other information processing device that has sent the request. Then, the web page for member registration is displayed on the other information processing device.
[0179] In addition, during the member registration process, the member information inputted in other information processing devices is received, and it is confirmed whether the inputted information is incomplete. If it is confirmed that the inputted information is incomplete, the user is prompted to re-enter the information. If there is no problem with the inputted information, the new member information is registered in the member database.
[0180] Furthermore, in the member registration process, a process of issuing an ID and a password in order to perform login for enabling various services and the like are also performed.
[0181] After executing the process of step S302 , the distribution server device 3 returns to the process of step S301 again.
[0182] When it is determined that no member registration request has been received in step S301, distribution server device 3 determines whether a tournament registration request has been received in step S303. In the case where it is determined that a tournament registration request has been received, distribution server device 3 executes a tournament registration process in step S304 as a response process.
[0183] In the tournament registration process, for example, web page data for applying for competition registration is sent to other information processing devices that have received the request. Then, the web page for applying for tournament registration is displayed on the other information processing devices.
[0184] In the web page for making a tournament registration application, for example, an operation of selecting a tournament to be registered for, or an operation of inputting a name of a player to be registered for, a player ID, etc. is possible. The distribution server device 3 receives the inputted pieces of information, and completes the tournament registration by registering the information in a database for managing players participating in the tournament.
[0185] Furthermore, when the entry to the tournament cannot be made due to the results or the like, web page data notifying that the entry cannot be made is transmitted to other information processing devices.
[0186] After executing the process of step S304, the distribution server device 3 returns to the process of step S301 again.
[0187] When it is determined that the tournament registration request has not been received in step S303, the distribution server device 3 determines whether a distribution content transmission request has been received in step S305. In the case where it is determined that the distribution content transmission request has been received, the distribution server device 3 performs distribution content transmission processing in step S306 as a response processing.
[0188] refer to Fig. 9 An example of distribution content sending processing will be described.
[0189] In step S401, the distribution server device 3 determines whether the user who requests the transmission of distribution content is in a logged-in state.
[0190] In a case where the user is not in the logged-in state, the distribution server device 3 performs a login process in step S402.
[0191] In the login process, an ID and a password are received from another information processing device used by the user who issued the request, and compared with the information stored in the login database to determine whether the login is permitted.
[0192] Whether the login is allowed is sent to the other information processing device that issued the request. Then, the login success / failure information is displayed in the other information processing device.
[0193] After the login process ends normally, or when it is determined in step S401 that the user is a user who is already in the logged-in state, in step S403, the distribution server device 3 receives distribution content selection information. The distribution content selection information is information for selecting a distribution content from a plurality of distribution contents, and is, for example, information designated by the user on a player, a tournament, and a match for which it is desired to send distribution content.
[0194] After receiving the distribution content selection information, the distribution server device 3 performs a process of acquiring the public range information of the designated distribution content in step S404. The public range information is, for example, information stored in a database in association with the distribution content.
[0195] In step S405, the distribution server device 3 determines whether the specified distribution content can be sent. The determination of whether to allow the sending is based on the requesting user who issued the sending request and the public range information. For example, if the user who issued the distribution request is the player himself, it is assumed that the requesting user is included in the public range and the sending of the distribution content is allowed.
[0196] Furthermore, when the requesting user is a family member of the player and the disclosure range is permitted because it also includes the family members, the sending of the distribution content is permitted.
[0197] On the other hand, when the requesting user is someone other than the player, and the disclosure range includes only the player himself or at most his team, it is assumed that the requesting user is not included in the disclosure range, and the sending of the distribution content is not permitted.
[0198] Note that, in the case where the disclosure scope is all users, sending of distribution content is permitted regardless of who the requesting user is.
[0199] If the transmission of the distribution content is permitted, the distribution server device 3 transmits the distribution content in step S406. On the other hand, if the transmission of the distribution content is not permitted, the distribution server device 3 transmits the fact that the transmission of the distribution content is not permitted to the requesting information processing device in step S407.
[0200] Description Return to Figure 8 .
[0201] After executing the distribution content sending process in step S306, the distribution server device 3 returns to the process in step S301, and determines whether various requests have been received.
[0202] If it is determined that the distribution content transmission request has not been received, the distribution server device 3 determines whether a guide information transmission request has been received in step S307. If it is determined that a guide information transmission request has been received, the distribution server device 3 performs guide information transmission processing in step S308 as a response processing.
[0203] refer to Fig.10 An example of guidance information transmission processing will be described.
[0204] In step S501, the distribution server device 3 determines whether the user who requested the transmission of the guide information is in a logged-in state, and executes the login processing of step S502 as necessary. These processes are similar to Fig. 9 The processing of steps S401 and S402 in .
[0205] When the login process ends normally, or when it is determined in step S501 that the user is a user who has logged in, in step S503, the distribution server device 3 receives guidance information selection information for selecting the type of guidance information. This information is information for specifying the guidance information desired by the user, and is information for specifying target performance including at least a tournament name, a match name, a player name, etc.
[0206] In addition, the guidance information selection information may include information for specifying items selected by the user from various items such as the entry position at the start, the number of breaths, the number of strokes, the number of kicks, the timing of starting to reach out for turning, the starting position of the turn, and the change in swimming speed.
[0207] By receiving these guide information selection information, the distribution server device 3 can grasp which information is to be transmitted from among a plurality of guide information.
[0208] In step S504, the distribution server device 3 acquires guide information based on the information received in step S503.
[0209] Then, in step S505, the distribution server device 3 transmits the guidance information.
[0210] Note that the disclosure range can be set in the guidance information. In this case, after step S503, only the process of determining whether the specified guidance information can be sent needs to be executed, for example, Fig. 9 Then, only when it is determined that the transmission is possible, the processing of steps S504 and S505 needs to be executed.
[0211] Description Return to Figure 8 .
[0212] After executing the guidance information transmission process of step S308, the distribution server device 3 returns to the process of step S301 again.
[0213] In step S309, the distribution server device 3 that has determined that no guide information transmission request has been received determines whether an entry information request has been received. The entry information request is issued by, for example, a player who plans to participate in a tournament and is a request to transmit various information related to the tournament to be participated in.
[0214] In the case where it is determined that a registration information request is received, in step S310, the distribution server device 3 performs registration information sending processing. Registration information sending processing is a process of sending necessary information to other information processing devices according to the user who issued the request. Note that from the perspective of personal information protection, the sending object of registration information can be limited. In this case, when a sending request is received from a non-target user, the registration information is not sent.
[0215] After receiving the entry information request or determining that the entry information request has not been received, the distribution server device 3 returns to the process of step S301.
[0216] The distribution server device 3 repeatedly executes Figure 8 The respective processes of steps S301, S303, S305, S307 and S309 illustrated in the figure are performed to determine whether various requests are received from other information processing devices and to appropriately perform response processing.
[0217] Note that except Figure 8In addition to what is shown in the diagram, the reception of a request to change player information and the reception of a request to quit can also be monitored.
[0218] <5. Live broadcast distribution>
[0219] The distribution server device 3 may have a function of performing live distribution (streaming distribution) on the day of the tournament. Fig.11 An example of processing performed by the distribution server device 3 for live broadcast distribution will be described.
[0220] In step S601, the distribution server device 3 determines whether the user requesting live broadcast distribution is in a logged-in state, and then in step S602, the distribution server device 3 appropriately performs a login process. These processes are similar to Fig. 9 The processing of steps S401 and S402 in .
[0221] After the requesting user enters the login state, the distribution server device 3 receives player selection information in step S603. The player selection information is information for selecting a player desired for live broadcast distribution from among players in the current competition, for example, specified by the requesting user.
[0222] After receiving the player selection information, the distribution server device 3 performs processing for combining captured image data in step S604. Thus, for example, Figure 4 and 5 The composite image data shown in the figure.
[0223] In step S605, the distribution server device 3 distributes the combined image data to each user. The distribution target terminal is, for example, the team terminal 4 or the personal terminal 5 used by the requesting user.
[0224] In step S606, the distribution server device 3 determines whether to end the live distribution. The live distribution ends, for example, when the competition ends. Alternatively, the live distribution ends when the user performs an operation for ending the distribution.
[0225] Without ending the live broadcast distribution, the distribution server device 3 performs the respective processes of steps S604 and S605 again.
[0226] For example, the processing of steps S604 and S605 is performed for each frame in the captured image data.
[0227] On the other hand, in the event that it is determined that the live broadcast distribution is to be ended, in step S607 , the distribution server device 3 performs processing to end the live broadcast distribution.
[0228] Note that for live broadcast distribution, it is desirable to use a line such as the fifth generation (5G) that is capable of high-speed, large-capacity, low-latency, and other connection communications compared to a line such as the fourth generation (4G). Thus, comfortable viewing is possible.
[0229] Furthermore, in some cases, it is preferable to distribute live broadcasts to users in the competition venue. For example, during a competition, the top players are often displayed on a large monitor installed in the venue, and people related to other players cannot see the images they want to see. Then, it is conceivable to watch the competition of the player you want to watch with the naked eye, but it may be difficult to watch due to the long distance from the player, or in the case of a swimming competition, it may be difficult to watch due to the influence of splashing water, etc. In this regard, it is conceivable to also distribute live broadcasts to users in the competition venue.
[0230] In the case of live broadcast distribution only to users in the venue, it is preferred to use a limited range, such as a local 5G (dedicated 5G) line built in the venue, a local 4G / LTE (dedicated 4G / LTE) line, etc. Thus, high-speed and secure distribution can be performed without using the network of the communication operator.
[0231] Note that for live distribution, different captured image data and synthesized image data are sent for each player, but the distribution objects can be limited. For example, the synthesized image data of a certain player can be distributed only to family members. In this case, after receiving the selection information of the distributed player in step S603, the process of determining whether the currently logged-in user is suitable as the distribution object can be performed, and only when it is determined that the currently logged-in user is suitable, the various processes from step S604 onwards are performed.
[0232] <6. Modifications>
[0233] <6-1. First Modification of Editing Example of Captured Image Data>
[0234] Although reference Figure 4 and 5 This section describes an example of editing captured image data, but here we will refer to Fig.12 Another editing example is described.
[0235] exist Fig.12 In the present invention, an image G1 for indicating a target record or a reference record is superimposed and displayed on an image obtained by imaging the player in the push mode. The image G1 is a line for indicating a target record or a reference record, and is displayed in front of, behind, etc., the advancing direction of the player of the subject. Alternatively, as the image G1, a line indicating the position of the player competing in the front position in the game may be used.
[0236] exist Fig.12 In the example illustrated in FIG. 1 , a line as an image G1 indicating a directory record of the player is displayed in front of the player, and a user browsing the distributed content can understand that the player is behind the target record by checking the positional relationship between the image G1 and the player.
[0237] In order to superimpose the image G1, first, it is necessary to calculate the superimposed position of the image G1. Specifically, the superimposed position of the image G1 is calculated based on the time taken for one round trip or the end time recorded such as the target record or the reference record, the current position of the player as the subject, and the time elapsed from the start. Thus, for example, a position 1 m in front of the player as the subject is calculated as the superimposed position.
[0238] Next, it is grasped which position in the image is the position 1 m ahead. To calculate the superimposed position of the image, for example, information acquired from the imaging device 7 that captures the captured image may be used, or image analysis may be performed.
[0239] In this way, the superimposed position of the image G1 is calculated.
[0240] The image generated by superimposing the image G1 is provided to the user as image content. Fig. 9 In the distribution content sending process of , the user can select the distribution content in which the image G1 is superimposed and displayed as in this example. Fig.10 In the guide information transmission process illustrated in FIG. 2 , the user can select, as guide information, image content in which the image G1 is superimposedly displayed as in this example.
[0241] Or, in Fig.11 In the live distribution process, the content with the image G1 superimposed thereon can be distributed in real time.
[0242] Note that Fig.12 In the example in which the image G1 is superimposed on the image obtained by imaging the player in the push mode, the image G1 may be superimposed and displayed on Figure 4 The first area AR1 or Figure 5Of course, the image G1 may be superimposed on both the image obtained by imaging the entire swimming pool displayed in the first area AR1 and the player's propulsion image displayed in the second area AR2. Furthermore, first, the image G1 is superimposed on the player's propulsion image displayed in the second area AR2, but in a case where the player's current position and the position based on the reference record or the target record deviate from each other so that the image G1 cannot be superimposed on the propulsion image displayed in the second area AR2, the image G1 may be switched to be superimposed on the overall image displayed in the first area AR1.
[0243] Note that instead of superimposing the image G1 indicating the target record or reference record, a plurality of images may be displayed. For example, images may be combined so that captured image data obtained by imaging a player's swimming at a personal best and captured image data of the same player in the current competition are displayed side by side. Thus, differences in breathing timing and differences in hand stroke timing may be found.
[0244] Of course, images can be combined so that captured image data of other players are displayed side by side. For example, a configuration can be created so that coaching information about a certain player can be obtained by combining captured image data obtained by imaging the swimming of the certain player and captured image data obtained by imaging the swimming of the world's top swimmers.
[0245] <6-2. Second Modification of the Editing Example of Captured Image Data>
[0246] In the above-described respective examples, a description is given of a case in which, for example, a video signal is generated by using captured image data acquired from the imaging device 7 installed in the venue. Figure 4 and 5 An example of the image content shown in the figure.
[0247] In the present modification, an example will be described in which image content is generated by using captured image data captured by an imaging device owned personally by a user.
[0248] Fig.13 An example of processing executed by the distribution server device 3 is illustrated.
[0249] In step S701, the distribution server device 3 determines whether a distribution content generation request is received. The request is a request for generation of distribution content using captured image data captured by a user personally.
[0250] In the case where no request is received, the distribution server device 3 performs the process of step S701 again.
[0251] On the other hand, in the case where it is determined that such a distribution content generation request is received, in step S702, the distribution server device 3 acquires the captured image data captured by the user. Then, the information processing apparatus such as the personal terminal 5 that sent the request performs a process of uploading the captured image data to the distribution server device 3. Note that the captured image data may be acquired together with the request reception of step S701.
[0252] Next, in step S703, the distribution server device 3 acquires information for specifying the target tournament. This information is sent from the personal terminal 5 or the like to the distribution server device 3, for example, when the user performs an operation of selecting the target tournament. Note that, instead of being received from the distribution server device 3, this information can be obtained by analyzing metadata contained in the uploaded captured image data or the like. Specifically, the target tournament or game can be specified based on the imaging date and time or global positioning system (GPS) information.
[0253] In step S704, distribution server device 3 acquires an entire image of the target game. The entire image is captured image data about the target game specified in step S703 among captured image data acquired by imaging device 7 installed in the venue, for example, an image obtained by imaging the entire swimming pool.
[0254] In step S705, the distribution server device 3 performs synchronization processing. When each captured image data used to generate the distribution content is acquired from the imaging device 7 set up in the venue, the captured image data can be synchronized by, for example, matching the time information of the imaging device 7. However, in this example, at least a part of the captured image data is captured by the user personally, and the imaging devices are not synchronized with each other.
[0255] Therefore, the synchronization process of step S705 is necessary.
[0256] In synchronization processing, it is conceivable that by performing image analysis on the acquired individual captured image data, the timing considered to be the same scene is extracted and synchronized based on the characteristic movements or splashes of the subject, or the movements of the audience or staff arranged by the swimming pool.
[0257] Alternatively, when each captured image data includes audio data, synchronization may be performed by analyzing the audio data.
[0258] After performing synchronization processing of each captured image data, the distribution server device 3 acquires text information in step S706. Figure 6 The processing is similar to the processing of step S104 in .
[0259] In step S707, the distribution server device 3 performs a combining process. This combining process is performed based on the processing result of the synchronization process performed in step S705. That is, for example, combining process is performed so that the Figure 4 Or the captured image data displayed in the first area AR1 and the captured image data displayed in the second area AR2 shown in the illustration in FIG. 5 .
[0260] By executing Fig.13 The distribution server device 3 can generate distribution content in which captured image data imaged by an individual is synchronized with captured image data imaged by an official, etc.
[0261] <7. Conclusion>
[0262] As described using various examples, the distribution server device 3 includes a content generation unit (content management unit 14), which generates display content (distribution content) by combining a first image captured by a first imaging device (imaging device 7) and a second image captured by the first imaging device (imaging device 7) or other imaging devices (which may be a personal terminal 5, etc.), wherein the second image has a part of the subject of the first image as a subject.
[0263] In the case of imaging a plurality of subjects, there is a need to simultaneously record an image of each subject or an image with a changed viewing angle, or a need for browsing.
[0264] For example, in the case of a swimming image, the image of each subject is a respective advancement image of a swimmer competing in the first lane and a swimmer competing in the eighth lane, or the like.
[0265] Furthermore, for example, in the case of a game image, the image whose angle of view is changed is an angle of view obtained by imaging the entire game, an angle of view for imaging only one player participating in the game, or the like.
[0266] However, it is difficult for individual users to retain such various types of images as recorded images due to imaging environments and photographers' deficiencies.
[0267] According to the above-mentioned various configurations, a display content is generated in which an official game image obtained by capturing the entire game and a game image of one player imaged by an individual are combined, thereby eliminating the need for an individual to prepare both equipment for imaging the entire game and equipment for imaging the player to be imaged. Furthermore, in the case where an official image is prepared for the image of the player to be imaged, imaging by an individual is not required, thereby improving convenience.
[0268] Furthermore, when the official match image obtained by capturing the entire match has a sufficiently high resolution, a progress image of the player to be captured can be generated by cutting out the entire image. Thus, the necessary imaging equipment can be reduced as much as possible, contributing to cost reduction.
[0269] Furthermore, by combining the overall image and the image of the individual player, it is possible to grasp the characteristics, improvement points, etc. of the individual player, which can be used to extract coaching information.
[0270] The distribution server device 3 may include a transmission processing unit (distribution control unit 19) that transmits display content.
[0271] Thus, the display content (for example, Figure 4 Or the distribution content shown in the diagram 5) is sent to the user terminal, etc.
[0272] As described in the second modification of the editing example of captured image data, the distribution server device 3 may include the synchronization processing unit 17 that performs synchronization processing of synchronizing the first image and the second image.
[0273] Thus, even when the first image and the second image are captured at different angles of view, the temporal relationship can be grasped.
[0274] For example, in a game image or the like, the overall image and the image of an individual player are synchronized so that the position of the player in the entire game or the like can be easily grasped.
[0275] As shown in the editing example of capturing image data ( Figure 4 and 5 ) as described in, the displayed content (distributed content) may include overlay information.
[0276] The superimposed information is, for example, character (text) information, image information, and the like.
[0277] When display contents are generated by superimposing character information and image information on a combination of a first image and a second image, the amount of information contained in the display contents can be increased, so that the content of the contents can be more deeply understood.
[0278] As shown in the editing example of capturing image data ( Figure 5 ), the overlay information may be descriptive information about the first image or the second image.
[0279] For example, in the case of a competition image, the description information is the date and time of the event, the venue of the event, the name of the competition, the names of the contestants, and the like.
[0280] By superimposing and displaying such information, for example, even when similar display contents exist, it is possible to easily grasp the captured contents.
[0281] As described in the first modification of the editing example of captured image data, the superimposition information may be comparison information on the subject of the second image.
[0282] The comparison information is, for example, information indicating a difference from a target record of a player who is a subject of the second image (eg, image G1 ). Specifically, the comparison information is information indicating how much the subject is ahead of or behind the target record.
[0283] By displaying such information in a superimposed manner, it is possible to understand the rhythm distribution of unknown players in the venue.
[0284] As described in the first modification of the editing example of captured image data, the comparison information may be information indicating the performance of the subject of the second image.
[0285] The performance of the subject of the second image is, for example, the past record of the player who is the subject of the second image.
[0286] That is, past personal best records and current records can be compared through images. In addition, for example, by superimposing images such as lines indicating personal best records, the excellent and poor parts of the game can be intuitively grasped, which can be used for future training, etc.
[0287] As described in the first modification of the editing example of captured image data, the comparison information may be information indicating the performance of a person other than the subject of the second image (eg, a world-class swimmer).
[0288] The results of people other than the subject of the second image may be, for example, the record of the best player among other players participating in the same competition, or the results may be results such as a Japanese record or a world record.
[0289] Therefore, the player's target performance and the performance of the current competition can be compared. In particular, as described above, by performing a line display based on the target record, the player's weaknesses can be intuitively grasped.
[0290] As described in the guidance information extraction process, an analysis processing unit 16 can be set up, which performs image analysis on the subject of the second image and extracts feature quantities that affect the performance of the subject of the second image, and the superimposed information (for example, text information displayed in the third area AR3) can be information related to the feature quantities.
[0291] For example, in the case of swimming, the feature quantities that affect the performance of the subject of the second image are the number or frequency of breathing, the number of hand strokes, the number of breathing, and the like.
[0292] By extracting, superimposing and displaying the feature quantities that affect performance, the player's movements can be analyzed more carefully, thereby obtaining guidance information for improving performance.
[0293] For example, in a distributed content delivery process ( Fig. 9 ) and other examples, an authentication processing unit 18 may be provided, the authentication processing unit 18 performs user authentication based on the received user information, and the sending processing unit (distribution control unit 19) may select and send display content (distribution content) based on the authentication result.
[0294] Display content is image content including personal information. Such content is only sent if the user authentication is successful.
[0295] Thus, personal information can be protected.
[0296] For example, in the distribution content sending process ( Fig. 9 ) as described above, a content management unit 14 may be provided, and the content management unit 14 sets a disclosure range for each display content (distribution content).
[0297] Depending on the subject player, the displayed content may or may not have viewing restrictions.
[0298] Since the disclosure range can be set, content management can be performed in consideration of the user's intention.
[0299] For example, in the distribution content sending process ( Fig. 9 ), an authentication processing unit 18 may be provided, the authentication processing unit 18 performs user authentication based on the received user information, and the sending processing unit (distribution control unit 19) may send display content (distribution content) based on the authentication result and the disclosure scope.
[0300] For example, the user is specified by user authentication, and whether to accept the user's request is determined based on the disclosure range set in the display content.
[0301] Therefore, it is possible to perform content management according to the user's intention and appropriately distribute content.
[0302] As shown in the second modification example of the captured image data editing example ( Fig.13 ), in the synchronization processing performed by the synchronization processing unit 17, an image comparison can be performed between a first image (for example, an image displayed in the first area AR1) and a second image (for example, an image displayed in the second area AR2).
[0303] For example, when the first image and the second image are moving images, the synchronization process is performed by partially comparing the two images with each other to calculate the degree of coincidence.
[0304] Thus, even if time stamp information is not associated with an image, two images can be synchronized. Furthermore, even if time stamp information is associated with an image, it is difficult to synchronize the two images based on the time stamp information when the images are captured by an unsynchronized imaging device. Even in this case, according to the present configuration, matching points can be searched by image processing and the matching points can be synchronized.
[0305] For example, in live broadcast distribution ( Fig.11 ) as described in the sending processing unit (distribution control unit 19) can perform streaming distribution.
[0306] For example, streaming distribution is performed by combining an official first image obtained by capturing an overall image in a bird's-eye view during a competition and an official second image focusing on a specific player.
[0307] Furthermore, by making the second image different for each player, display content that is different for each player is streamed and distributed. Thus, when the desired image is different for each user, display content suitable for each user can be streamed and distributed.
[0308] As described above, 5G / local 5G (or local 4G / LTE) lines can be used in live broadcast distribution. In this case, a network slicing technology can be used in which the network is virtualized, network resources are divided (sliced), and the slices are appropriately used according to the purpose and priority of the information to be transmitted.
[0309] Specifically, for example, the distribution server device 3 is installed in the venue, builds a local 5G communication environment in the venue, and controls the base station of the communication to distribute various data and image data to each terminal to be distributed. In this case, for example, each terminal can perform processing of combining image data or superimposing various data on image data.
[0310] At this time, distribution server device 3 generates information for specifying slices to be distributed according to the purpose and characteristics of data to be distributed, and transmits the data and information (metadata) for specifying the slices to which the signal is to be distributed to the base station in association with each other.
[0311] The base station allocates each reception data to the slices specified by each metadata, and transmits each data to each terminal to be distributed by using the allocated slices.
[0312] Here, as an example of slice allocation, it is conceivable that image data that is preferably viewed without delay is allocated to slices for low-delay transmission. In addition, in the case where the delay of image data is allowed, such as a case where visual game viewing is mainly performed, for example, it is conceivable that the image data is allocated to slices for large-capacity transmission, and various information superimposed on the image data that is preferably grasped in a timely manner (for example, progress information such as the time taken for one round trip) is allocated to slices for low-delay transmission.
[0313] Note that in the above example, a swimming competition is described as an example, but other competitions can also be used as objects. For example, the object can be a track and field competition such as a 100m sprint or a 110m hurdles, or a team competition such as synchronized swimming. In addition, in addition to competitions, the present invention can also be applied to wind band concerts, orchestra concerts, etc.
[0314] Specifically, in the case of a group competition, captured image data obtained by imaging all members of the group can be used as an overall image, captured image data obtained by imaging one of the contestants among the members in a propulsion mode can be used as a propulsion image, and the captured image data can be combined to generate distribution content.
[0315] Alternatively, in the case of a wind band concert, captured image data obtained by imaging all performers can be used as an overall image, captured image data obtained by imaging one of the performers in a push mode can be used as a push image, and the captured image data can be combined to generate distribution content.
[0316] That is, the above-mentioned various structures can be applied to performances or competitions performed by multiple people.
[0317] The program according to the embodiment is used to cause a CPU, a DSP, etc., or a device including a CPU or a DSP to execute Figure 6 , 7 , 8, 9, 10, 11 and 13 each illustrate the procedures for the respective processing.
[0318] That is, the program according to the embodiment causes the control unit of the distribution server device 3 to perform generation processing, which generates display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, wherein the second image takes a part of the subject in the first image as a subject.
[0319] By such a program, the above-mentioned distribution server device 3 can be realized.
[0320] Such a program for realizing the distribution server device 3 may be recorded in advance in a HDD as a recording medium built in a device such as a computer device, a ROM in a microcomputer having a CPU, or the like.
[0321] Alternatively, the program may be temporarily or permanently stored (recorded) in a removable recording medium such as a floppy disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card. Such a removable recording medium may be provided as so-called packaged software.
[0322] Furthermore, in addition to installing such a program from a removable recording medium to a personal computer or the like, the program may also be downloaded from a download site via a network such as a local area network (LAN) or the Internet.
[0323] In addition, such a program is suitable for the wide provision of the distribution server device 3 of the embodiment. For example, when the program is downloaded to a portable terminal device such as a smartphone or a tablet, or a device with a camera function such as a mobile phone, a personal computer, a game device, a video device, or a personal digital assistant (PDA), the smartphone or the like can function as the distribution server device 3 of the present disclosure.
[0324] Note that the effects described in this specification are merely examples and are not limiting, and other effects may be provided.
[0325] <8. This technology>
[0326] (1) An information processing device comprising:
[0327] A content generating unit generates display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, the second image having a part of a subject in the first image as a subject.
[0328] (2) The information processing device according to (1), further comprising:
[0329] A sending processing unit is used to send display content.
[0330] (3) The information processing device according to (1) or (2), further comprising:
[0331] A synchronization processing unit performs synchronization processing to synchronize the first image and the second image.
[0332] (4) The information processing device according to any one of (1) to (3),
[0333] The displayed content includes overlay information.
[0334] (5) The information processing device according to (4),
[0335] The superimposed information is description information about the first image or the second image.
[0336] (6) The information processing device according to (4) or (5),
[0337] The superimposed information is comparison information of the subject of the second image.
[0338] (7) The information processing device according to (6),
[0339] The comparison information is information indicating the performance of the subject of the second image.
[0340] (8) The information processing device according to (6) or (7),
[0341] The comparison information is information indicating the performance of a person other than the subject of the second image.
[0342] (9) The information processing device according to any one of (4) to (8), further comprising:
[0343] an analysis processing unit that performs image analysis on a subject of the second image and extracts a feature quantity that affects a performance of the subject of the second image,
[0344] The superimposed information is information related to the feature quantity.
[0345] (10) The information processing device according to (2), further comprising:
[0346] an authentication processing unit, the authentication processing unit performing user authentication based on the received user information,
[0347] The transmission processing unit selects and transmits display content based on the authentication result.
[0348] (11) The information processing device according to (2), further comprising:
[0349] A content management unit is provided for setting a disclosure range for each display content.
[0350] (12) The information processing device according to (2), further comprising:
[0351] an authentication processing unit, the authentication processing unit performing user authentication based on the received user information,
[0352] The transmission processing unit transmits the display content based on the authentication result and the disclosure range.
[0353] (13) The information processing device according to (3),
[0354] In the synchronization process, an image comparison is performed between the first image and the second image.
[0355] (14) The information processing device according to (2), (10), (11) or (12),
[0356] The sending processing unit performs streaming distribution.
[0357] (15) An information processing method executed by an information processing device, the method comprising:
[0358] The display content is generated by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, the second image having a part of the subject in the first image as a subject.
[0359] (16) A program for causing an information processing device to execute:
[0360] A generation process of generating display content by combining a first image captured by a first imaging device and a second image captured by the first imaging device or another imaging device, the second image having a part of a subject in the first image as a subject.
[0361] Reference numerals list
[0362] 3 Distribution Server Equipment
[0363] 7 Imaging equipment
[0364] 14 Content Management Unit
[0365] 16 Analysis and processing unit
[0366] 17 Synchronous Processing Unit
[0367] 18 Authentication Processing Unit
[0368] 19 Distribution control unit (sending processing unit)
Claims
1. An information processing device, comprising: A content generation unit that generates display content by combining a first image of a sports event captured by a first imaging device, a second image captured by the first imaging device or another imaging device, and superimposed information, wherein The second image takes a part of the subject in the first image as the subject. The superimposed information is comparative information indicating the performance of the subject of the second image, the comparative information being obtained by image analysis of a third image of an electronic bulletin board displayed during a sports event captured by the first imaging device or another imaging device; A sending processing unit, wherein the sending processing unit sends display content; as well as A synchronization processing unit performs synchronization processing to synchronize the first image and the second image.
2. The information processing device according to claim 1, The superimposed information is description information about the first image or the second image.
3. The information processing device according to claim 1, Here, the comparison information is information indicating the performance of a person other than the subject of the second image.
4. The information processing device according to claim 1, further comprising: an analysis processing unit that performs image analysis on a subject of the second image and extracts a feature quantity that affects a performance of the subject of the second image, The superimposed information is information related to the feature quantity.
5. The information processing device according to claim 1, further comprising: an authentication processing unit, the authentication processing unit performing user authentication based on the received user information, The transmission processing unit selects and transmits display content based on the authentication result.
6. The information processing device according to claim 1, further comprising: A content management unit is provided for setting a disclosure range for each display content.
7. The information processing device according to claim 6, further comprising: an authentication processing unit, the authentication processing unit performing user authentication based on the received user information, The transmission processing unit transmits the display content based on the authentication result and the disclosure range.
8. The information processing device according to claim 1, In the synchronization process, an image content comparison is performed between the first image and the second image.
9. The information processing device according to claim 1, The sending processing unit performs streaming distribution.
10. An information processing method performed by an information processing device, the method comprising: The display content is generated by combining a first image of the sporting event captured by a first imaging device, a second image captured by the first imaging device or another imaging device, and overlay information, wherein The second image takes a part of the subject in the first image as the subject. the superimposed information is comparative information indicating the performance of the subject of the second image, the comparative information being obtained by image analysis of a third image of an electronic bulletin board displayed during a sports event captured by the first imaging device or another imaging device; Send display content; as well as A synchronization process of synchronizing the first image and the second image is performed.
11. A program product, comprising a program, which, when executed by an information processing device, causes the information processing device to execute: A generation process that generates display content by combining a first image of a sporting event captured by a first imaging device, a second image captured by the first imaging device or another imaging device, and overlay information, wherein The second image takes a part of the subject in the first image as the subject. the superimposed information is comparative information indicating the performance of the subject of the second image, the comparative information being obtained by image analysis of a third image of an electronic bulletin board displayed during a sports event captured by the first imaging device or another imaging device; Sending process, wherein the sending process sends the display content; as well as A synchronization process is performed to synchronize the first image with the second image.
12. A non-transitory readable storage medium having a program stored thereon, wherein when the program is executed by an information processing device, the information processing device is caused to execute: A generation process that generates display content by combining a first image of a sporting event captured by a first imaging device, a second image captured by the first imaging device or another imaging device, and overlay information, wherein The second image takes a part of the subject in the first image as the subject. the superimposed information is comparative information indicating the performance of the subject of the second image, the comparative information being obtained by image analysis of a third image of an electronic bulletin board displayed during a sports event captured by the first imaging device or another imaging device; Sending process, wherein the sending process sends the display content; as well as A synchronization process is performed to synchronize the first image with the second image.
