Information processing device, program, and information processing method

Abstract

To reduce the amount of data while allowing a user to select a desired screen in viewing a video content composed of a plurality of screens.SOLUTION: An information processing apparatus includes: means of acquiring second video data including a plurality of screens corresponding to multiple pieces of first video data, respectively; means of acquiring a user instruction to designate one of the screens, as a designated screen; means of enlarging the second video data; means of correcting positions of the screens so that the designated screen may be located in a first display area and screens not designated by the user instruction may be located outside the first display area; and means of displaying the designated screen included in the second video data which has been enlarged and whose position has been corrected, in the first display area.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] In recent years, the demand for online seminars has been increasing. In an online seminar, it is common to watch multiple input videos (for example, videos of presentation materials and a video of a lecturer). A viewer can view a desired input moving image by specifying the desired input moving image.

[0003] For example, Patent Document 1 discloses a technique for improving operability by reducing the effort required to select a desired image and display it on a display unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-185563 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in order to switch between multiple input videos, video data corresponding to each of the multiple input videos (i.e., multiple video data) is required. This results in an extremely large amount of data. The more videos to be switched, the larger the data amount becomes. The increase in data amount reduces the quality of the viewing experience, especially when watching live broadcasts and in poor communication environments.

[0006] In Patent Document 1, an image specified by a user is selectively displayed from among a plurality of images, so it is not possible to suppress an increase in the amount of data.

[0007] An object of the present invention is to suppress the amount of data while allowing a user to switch between desired screens when viewing video content made up of a plurality of screens. [Means for solving the problem]

[0008] One aspect of the present invention is means for acquiring second video data including a plurality of screens corresponding to the plurality of first video data, means for acquiring a user instruction to designate one of the plurality of screens as a designated screen; means for enlarging the second video data; a means for correcting positions of the plurality of screens so that the designated screen is located within a first display area, and non-designated screens that are not designated by the user instruction are located outside the first display area; a display unit for displaying, in the first display area, a designated screen included in the enlarged and position-corrected second video data; It is an information processing device. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a functional block diagram of the information processing system of FIG. 1. [Diagram 3] FIG. 2 is a functional block diagram of the distribution system of FIG. [Figure 4] FIG. 1 is an explanatory diagram of an overview of the present embodiment. [Diagram 5] FIG. 4 is a sequence diagram of a distribution process according to the present embodiment. [Figure 6] 6 is a diagram showing an example of a screen displayed in the process of FIG. 5. [Figure 7] FIG. 6 is an explanatory diagram of the image processing of FIG. 5. [Figure 8] FIG. 13 is a diagram showing an example of a screen displayed in another modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings for explaining the embodiment, the same components are generally designated by the same reference numerals, and the repeated description will be omitted.

[0011] (1) Information Processing System Configuration The configuration of the information processing system will be described. Fig. 1 is a block diagram showing the configuration of the information processing system of this embodiment. Fig. 2 is a functional block diagram of the information processing system of Fig. 1. Fig. 3 is a functional block diagram of the distribution system of Fig. 1.

[0012] As shown in FIG. 1, the information processing system 1 includes a client device 10, a server 30, and a distribution system 50. The client device 10, the server 30, and the distribution system 50 are connected via a network (eg, the Internet or an intranet) NW.

[0013] The client device 10 is a computer (an example of an "information processing device") that transmits a request to the server 30. The client device 10 is, for example, a smartphone, a tablet terminal, or a personal computer.

[0014] The server 30 is a computer (an example of an "information processing device") that provides the client device 10 with a response in response to a request transmitted from the client device 10. The server 30 is, for example, a web server.

[0015] The distribution system 50 is a system that distributes moving images to the client device 10 via the server 30 (an example of an “information processing device”).

[0016] (1-1) Client device configuration The configuration of the client device 10 will be described.

[0017] As shown in FIG. 2, the client device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14.

[0018] The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of a Read Only Memory (ROM), a Random Access Memory (RAM), and a storage (for example, a flash memory or a hard disk).

[0019] The programs include, for example, the following programs: ·OS (Operating System) programs · Programs for applications that process information (e.g. web browsers)

[0020] The data includes, for example, the following data: Databases referenced in information processing Data obtained by performing information processing (i.e., the results of performing information processing)

[0021] The processor 12 is configured to realize the functions of the client device 10 by starting a program stored in the storage device 11. The processor 12 is, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

[0022] The input / output interface 13 is configured to obtain user instructions from an input device connected to the client device 10 , and to output information to an output device connected to the client device 10 . The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display.

[0023] The communication interface 14 is configured to control communications between the client device 10 and the server 30 .

[0024] (1-2) Server configuration The configuration of the server 30 will now be described.

[0025] As shown in FIG. 2, the server 30 includes a storage device 31, a processor 32, an input / output interface 33, and a communication interface .

[0026] The storage device 31 is configured to store programs and data, and is, for example, a combination of a ROM, a RAM, and a storage (for example, a flash memory or a hard disk).

[0027] The programs include, for example, the following programs: -OS programs Application programs that perform information processing

[0028] The data includes, for example, the following data: Databases referenced in information processing Results of information processing

[0029] The processor 32 is configured to realize the functions of the server 30 by running a program stored in the storage device 31. The processor 32 is, for example, a CPU, an ASIC, an FPGA, or a combination of these.

[0030] The input / output interface 33 is configured to receive user instructions from an input device connected to the server 30 , and to output information to an output device connected to the server 30 . The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display.

[0031] The communication interface 34 is configured to control communications between the server 30 and the client device 10 .

[0032] (1-3) Distribution system configuration The configuration of the distribution system 50 will be described.

[0033] As shown in FIG. 3, the distribution system 50 includes a computer 51, cameras 52 to 54, a screen synthesizer 55, microphones 56 to 57, an audio mixer 58, and a distribution device 59.

[0034] The system includes a computer 51, cameras 52 to 54, a screen synthesizer 55, microphones 56 to 57, an audio mixer 58, and a distribution device 59.

[0035] The computer 51 has a similar configuration to the client device 10. The computer 51 is configured to output first video data in which a screen displayed on a display (e.g., a screen of a presentation material) is recorded. The computer 51 is, for example, a smartphone, a tablet terminal, or a personal computer.

[0036] Cameras 52 to 54 are configured to capture moving images and generate moving image data of the moving images (hereinafter referred to as "first moving image data").

[0037] Screen synthesizer 55 is configured to receive a plurality of first moving image data from computer 51 and cameras 52 to 54, and to convert the plurality of first moving image data into one moving image data (hereinafter referred to as "second moving image data").

[0038] The microphones 56 to 57 are configured to input voice and output voice data.

[0039] The audio mixer 58 is configured to mix the audio data output from the microphones 56 to 57 with the output of the screen mixer 55 (second moving image data).

[0040] The distribution device 59 is configured to transmit the output of the audio mixer 58 (the second moving image data with which the audio data has been mixed) to the server 30.

[0041] (2) Overview of the embodiment An overview of this embodiment will be described below with reference to Fig. 4.

[0042] As shown in FIG. 4, a computer 51 outputs first video data of a presentation material. The camera 52 outputs first moving image data of the lecturer during the presentation. The camera 53 outputs first video data of the lecturer during the presentation and the screen on which the presentation materials are projected. The camera 54 outputs first video data of a whiteboard used by a lecturer during a presentation. The image synthesizer 55 is configured to synthesize a plurality of first moving image data output from the computer 51 and the cameras 52 to 54, thereby converting the data into one second moving image data. The microphones 56 to 57 output the voice data of the lecturer during the presentation. The audio mixer 58 combines the multiple pieces of audio data output from the microphones 56 to 57 with the second moving image data. The distribution device 59 transmits the output of the audio mixer 58 (the second moving image data in which the multiple audio data are mixed) to the server 30.

[0043] The viewer uses client device 10 to view the second video data transmitted from distribution device 59 by switching between first screen A corresponding to the first video data output by computer 51, second screen B corresponding to the first video data output by camera 52, second screen C corresponding to the first video data output by camera 53, and second screen D corresponding to the first video data output by camera 54.

[0044] (3) Distribution process The distribution process of this embodiment will be described. Fig. 5 is a sequence diagram of the distribution process of this embodiment. Fig. 6 is a diagram showing an example of a screen displayed in the process of Fig. 5. Fig. 7 is an explanatory diagram of the image processing of Fig. 5.

[0045] An example of live distribution of the second moving image data output by the distribution system 50 will be described below.

[0046] As shown in FIG. 5, the distribution system 50 executes shooting (S150). Specifically, the image synthesizer 55 synthesizes the first moving image data (FIG. 4) output by the computer 51 and the cameras 52 to , thereby generating the second moving image data. The audio mixer 58 mixes the audio output from the microphones 56 to 57 with the second moving image data. The distribution device 59 transmits the output of the audio mixer 58 (that is, the second moving image data with the audio mixed therein) to the server 30.

[0047] After step S150, the server 30 executes the distribution of the second video data (S130). Specifically, the processor 32 acquires the second video data from the distribution device 59, and starts distributing the second video data.

[0048] After step S130, the client device 10 executes screen display (S110). Specifically, when the user gives a user instruction to view the second video data (e.g., an instruction to access a live streaming website) to the input / output interface 13, the processor 12 displays screen P10 (FIG. 6) on the display.

[0049] The screen P10 includes a first display area A10a, a second display area A10b, and operation objects B10a to B10e.

[0050] Second video data MV10a is displayed in first display area A10a. The second video data MV10a includes images corresponding to the following first video data. First video data output by the computer 51 (i.e., video of presentation materials) First video data output by the camera 52 (i.e., a video of the lecturer during a presentation) First video data output by the camera 53 (i.e., video of the lecturer and the screen during the presentation) First video data output by the camera 54 (i.e., a video of the whiteboard used by the lecturer during the presentation) That is, the image size of the second video data is a size that allows a plurality of first video data (for example, four first video data) to be allocated to the first display area A10a.

[0051] The second display area A10b is smaller than the first display area A10a. In the second display area A10b, second video data MV10b, which is smaller than the first display area A10a, is always displayed.

[0052] The operation object B10a is an object that accepts a user instruction for displaying the second video data in the first display area A10a.

[0053] The operation object B10b is an object that receives a user instruction for designating, as a designated screen in the first display region A10a, a screen corresponding to the first moving image data (moving image of the presentation material) output from the computer 51, out of the second moving image data MV10a. First region information indicating the first moving image data output from the computer 51 is assigned to the operation object B10b.

[0054] The operation object B10c is an object that accepts a user instruction to designate, as a designated screen, a screen corresponding to the first moving image data (video of the lecturer during a presentation) output from the camera 52 out of the second moving image data MV10a in the first display region A10a. The second region information indicating the first moving image data output from the camera 52 is assigned to the operation object B10b.

[0055] The operation object B10d is an object that receives a user instruction to specify, as a specified screen in the first display region A10a, a screen corresponding to the first moving image data (moving image of the lecturer during the presentation and a screen on which the presentation materials are projected) of the second moving image data MV10a output from the camera 53. The operation object B10d is assigned third region information indicating the first moving image data output from the camera 53.

[0056] The operation object B10e is an object that accepts a user instruction to specify, as a specified screen in the first display region A10a, a screen corresponding to the first moving image data (a whiteboard used by the lecturer during a presentation) output from the camera 54 out of the second moving image data MV10a. Fourth region information indicating the first moving image data output from the camera 54 is assigned to the operation object B10e.

[0057] After step S110, the client device 10 performs enlargement and position correction (S111). Specifically, when the user operates the operation object B10b on the screen P10, as shown in FIG. 7, the processor 12 enlarges the second video data MV10a so that the first screen MV10aa corresponding to the first region information in the second video data MV10a is the same size as the first display region A10a. Processor 12 corrects the display position of second video data MV10a so that the enlarged first screen MV10aa is included in first display region A10a. As a result, non-designated screens of second video data MV10a other than the first screen MV10aa (i.e., second screen MV10ab corresponding to the second region information to fourth screen MV10ad corresponding to the fourth region information) are moved out of first display region A10a. In this way, when changing from a state in which all screens are displayed to a screen, enlargement and position correction are performed.

[0058] The process of step S111 is similar to that of the case where any one of the operation objects B10b to B10d is operated in step S110.

[0059] After step S111, the client device 10 executes screen display (S112). Specifically, the processor 12 displays a screen P11 (FIG. 6) on the display.

[0060] On screen P11, a first screen MV10aa (an example of a "first specified screen") is displayed in the entire first display area A10a (that is, the second screen MV10ab to the fourth screen MV10ad are not displayed).

[0061] After step S112, the client device 10 performs position correction (S113). Specifically, when the user operates the operation object B10e, the processor 12 corrects the display position of the second video data MV10a so that the enlarged fourth screen MV10ad (an example of a "second designated screen") is included in the first display area A10a, as shown in Fig. 7. As a result, non-designated screens of the second video data MV10a other than the fourth screen MV10ad (i.e., the first screen MV10aa to the third screen MV10ac) are moved out of the first display area A10a. In this way, when changing from a state in which one screen is displayed to another screen, position correction is performed.

[0062] The process of step S113 is similar to that of the case where the operation object B10b or B10c is operated in step S112.

[0063] After step S113, the client device 10 executes screen display (S114). Specifically, the processor 12 displays a screen P12 (FIG. 6) on the display.

[0064] On screen P12, the fourth screen MV10ad is displayed in the entire first display area A10a (that is, the first screen MV10aa to the third screen MV10ac are not displayed).

[0065] After step S114, the client device 10 cancels the enlargement and position correction (S115). Specifically, when the user operates the operation object B10a, as shown in FIG. 7, the processor 12 cancels the enlargement and position correction of the second video data MV10a executed in step S111.

[0066] After step S115, the client device 10 executes screen display (S116). Specifically, the processor 12 displays screen P10 (FIG. 6) on the display. Since the enlargement and position correction are canceled in step S115, in step S116, a screen corresponding to the second video data MV10a including a plurality of first video data is displayed, similar to step S110.

[0067] (4) Summary of the present embodiment The client device 10 of this embodiment enlarges, reduces, or corrects the position of one piece of second video data MV10a including a plurality of screens, in accordance with the position of a specified screen designated by the user. This makes it possible to reduce the amount of data while allowing the user to switch between desired screens when viewing video content made up of multiple screens.

[0068] Furthermore, the client device 10 of the present embodiment may enlarge or reduce the second video data MV10a so that the size of the first display area A10a matches the size of the designated screen. This allows the user to view the specified screen at the maximum size within the range of the first display region A10a.

[0069] Furthermore, the client device 10 of the present embodiment may display the second video data MV10a in a second display area A10b that is smaller than the first display area A10a. In this case, a plurality of screens are simultaneously displayed in the second display area A10b. This allows the user to switch between screens displayed in the first display area A10a while viewing multiple screens in the second display area A10b.

[0070] (5) Other modifications Other modifications will now be described.

[0071] The storage device 11 may be connected to the client device 10 via a network NW. The storage device 31 may be connected to the server 30 via the network NW.

[0072] In the present embodiment, an example of live distribution of the second moving image data has been described, but the present embodiment is not limited to this. The present embodiment can also be applied to an example of viewing the second moving image data stored in the storage device 11. In this case, the processor 12 stores in the storage device 11 the second moving image data output from the distribution device 59 in step S150. The processor 12 performs the processes of steps S110 to S116 on the second moving image data stored in the storage device 11.

[0073] In the present embodiment, an example has been shown in which the first moving image data generated by the computer 51 and the cameras 52 to 54 and the audio data generated by the microphones 56 to 57 are synthesized, but the present embodiment is not limited to this. The present embodiment can also be applied to any of the following examples. Example that does not include audio data Example of the first video data being generated by multiple computers -Examples where the number of cameras generating the first video data is other than three

[0074] In the present embodiment, an example in which the number of first moving image data included in the second moving image data is four has been shown, but the present embodiment is not limited to this. The present embodiment is also applicable to an example (FIG. 8) in which the number of first moving image data included in the second moving image data is any number equal to or greater than two (for example, three).

[0075] In this embodiment, an example has been shown in which the display sizes of multiple first moving image data included in the second moving image data are uniform, but this embodiment is also applicable to an example in which the display sizes of multiple first moving image data included in the second moving image data are different from each other (Figure 8).

[0076] Although the embodiment of the present invention has been described in detail above, the scope of the present invention is not limited to the above embodiment. Furthermore, the above embodiment can be improved or modified in various ways without departing from the spirit of the present invention. Furthermore, the above embodiment and the modified examples can be combined. [Explanation of symbols]

[0077] 1: Information processing system 10: Client device 11:Storage device 12: Processor 13: Input / Output Interface 14: Communication interface 30: Server 31: Storage device 32 : Processor 33: Input / Output Interface 34: Communication interface 50: Distribution system 51: Computer 52~54: Camera 55:Screen synthesis device 56~57: Mike 58: Audio Mixer 59: Distribution device

Claims

1. 1. An information processing device that controls display of a first display area of ​​a display and display of a second display area that is smaller than the first display area and is disposed at a position not overlapping with the first display area, A means for acquiring second moving image data including a plurality of first moving image data, a means for acquiring a user instruction to designate one of the first screens corresponding to the plurality of first video data as a designated screen; means for enlarging the second video data so that the designated screen has the same size as the first display area; a means for correcting a display position of the second video data so that, in the enlarged second video data, the designated screen is included in a first display area, and a non-designated screen, which is a first screen not designated by the user instruction, is moved out of the first display area; a display unit for displaying the enlarged and position-corrected designated screen in the second moving image data in the first display area; a display unit for displaying a first screen corresponding to each of the plurality of first video data in the second display area; Information processing device.

2. the correcting means, when a user instruction to designate the non-designated screen as a second designated screen is acquired while a first designated screen is being displayed, corrects the display position of the second video data so that the second designated screen is included in a first display area and non-designated screens other than the second designated screen are removed from the first display area. The information processing device according to claim 1 .

3. a canceling unit configured to cancel the enlargement and the position correction when a user instruction for designating a screen corresponding to the plurality of first video data is acquired while the designated screen is being displayed; the display means displays a plurality of first screens corresponding to the second video data in the first display area.

3. The information processing device according to claim 1 or 2.

4. The second video data includes first video data of a lecturer giving a presentation, first video data of the lecturer and a screen on which the presentation materials are projected, first video data of a whiteboard used by the lecturer, and first video data of the presentation materials.

4. The information processing device according to claim 1.

5. A program for causing a computer to function as any one of the means according to claims 1 to 4.

6. 1. An information processing method for controlling, by a computer, display of a first display area of ​​a display and display of a second display area that is smaller than the first display area and is displayed at a position not overlapping with the first display area, comprising: The computer executes a step of acquiring second moving image data including a plurality of first moving image data; The computer executes a step of acquiring a user instruction to designate one of the first screens corresponding to the plurality of first video data as a designated screen; the computer executes a step of enlarging the second video data so that the designated screen has the same size as the first display area; the computer executes a step of correcting a display position of the second video data so that, in the enlarged second video data, the designated screen is included in a first display area, and a non-designated screen, which is a first screen not designated by the user instruction, is moved out of the first display area; The computer executes a step of displaying a designated screen in the enlarged second video data whose position has been corrected in the first display area; displaying, in the second display area, a first screen corresponding to each of the plurality of first video data; Information processing methods.

Images