Information processing device, information processing program, and recording medium

Abstract

To provide an information processing device suitable for switching and distributing videos from a plurality of video sources including a network camera via HTTP streaming.SOLUTION: An information processing device 100 for performing information processing for distributing a video content using, as raw materials, videos of a plurality of video sources including a network camera 1500 includes video source thumbnail display means for generating and displaying a video source thumbnail based on a video signal transferred by a transfer method in which a node for transmitting the video source transfers data without confirming reception by a receiving node, and a switching means, and further includes play list generating means for generating a play list based on information regarding which video source was switched at what timing, and information processing means for distributing the video content to a viewer terminal 1600 by a transfer method in which the same data is repeatedly re-transmitted until a reception notification of the receiving node is confirmed.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device that processes information related to the distribution of video content made up of video images from a plurality of video sources. [Background technology]

[0002] Generally, video content for TV programs and the like is created by displaying on a monitor screen video signals captured by multiple imaging devices from different directions (angles of view) of the subject, and then using a switcher (video switching means) to switch (switch) between the video images to be used in the video content provided to the viewer and the video images that are not to be used by operating a switch button or the like.

[0003] Meanwhile, network cameras that transmit video signals via computer networks such as the Internet and LANs have become widespread. In recent years, video cameras and digital cameras for business and broadcasting use and general photography have also appeared that are capable of transmitting the images they capture via computer networks.

[0004] Therefore, there is a need and challenge to use such network cameras (cameras that can transmit video via a computer network) to create video content by switching video sources, thereby reducing the effort required for line setup and equipment costs, thereby making it easier, faster, and less costly to create video content.

[0005] In recent years, the mainstream of video distribution on the Internet is HTTP streaming, which transmits video in compliance with HTTP (HyperText Transfer Protocol). HTTP streaming segments (subdivides) video data into Transport Stream (TS) files of very short duration (for example, 10 seconds), transfers the HTTP data together with a playlist (formats include M3U files and MPD files) that instructs playback, and plays the received TS files continuously on the viewer's terminal according to the playlist. HTTP streaming includes formats such as HLS (HTTP Live Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP).

[0006] There are two methods for transmitting data on a computer network: using TCP (Transmission Control Protocol), which continues to resend the same data (packet) until the receiving node acknowledges receipt, and using UDP (User Datagram Protocol), which allows the sending node to transfer data without acknowledging receipt by the receiving node. HTTP streaming uses TCP (Transmission Control Protocol) for data transfer, which reduces degradation of signal quality due to data transfer. In addition, it is the same transfer method as general file transfers on web servers, which can reduce distribution costs.

[0007] Patent Document 1 discloses a camera system technology that has a plurality of camera devices that packetize and output captured video signals, and a relay device connected to the plurality of camera devices and relays the video signals transmitted from the plurality of camera devices, the relay device having a receiving unit that receives each video signal synchronized between the plurality of camera devices based on a synchronization signal, and a switch unit that selects and outputs the video signals output from the plurality of camera devices based on a control signal for selecting a video, thereby making it possible to select a desired video from among videos from the plurality of camera devices and simplifying the system configuration.

[0008] Patent Document 2 also discloses technology for a video transmission device and monitor system that can be applied to an IP-based broadcasting system, by comprising a video transmission device that receives an uncompressed video signal and generates an IP packet stream of uncompressed video data, while at the same time compressing the uncompressed signal and generating an IP packet stream of compressed video data; a video distribution system that distributes an IP packet stream of uncompressed video data from among a plurality of IP packet streams received from the video transmission devices as high-quality video, while at the same time distributing an IP packet stream of compressed video data to a monitor system for video monitoring; and a monitor system that selects an IP packet stream to be displayed on a monitor from the IP packet stream of compressed video data and displays it on the monitor.

[0009] Patent Document 3 discloses a technology for an information processing device that can determine the composition and how to switch between video sources including multiple cameras by looking at text (characters and sentences) that indicate the content structure, such as a script or interview notes, by allocating video sources including imaging devices and camera work to portions of input text.

[0010] Patent Document 4 also discloses technology for a video streaming distribution device that generates a clip information file that identifies a video clip for playback extracted from streaming data and a playback instruction file for instructing the playback of multiple segment files that make up the clip, edits the clip information file including the playback instruction file, and transfers the clip information file to a public site, thereby making it possible to publish on the Web a clip information file including a playlist that defines clips corresponding to highlight scenes while live streaming the video, for example in a live sports broadcast.

[0011] Patent document 5 discloses a content distribution technology that includes a playback means for playing a video file including a plurality of TS packets, a UDP packet receiving means for decomposing a video file including a plurality of TS packets and receiving UDP packets transmitted via IP multicast, and a video file reconstruction means for reconstructing the video file from the UDP packets received by the UDP packet receiving means and supplying the video file to the playback means in accordance with HTTP, wherein the video file reconstruction means reconstructs the video file in a manner including only complete TS packets when at least one UDP packet of the plurality of UDP packets corresponding to the video file is not received, thereby using IP multicast communication when distributing video in accordance with HTTP and further reducing playback loss as much as possible. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] JP 2015-62311 A [Patent Document 2] JP 2016-9881 A [Patent Document 3] JP 2012-14420 A [Patent Document 4] Patent No. 5909546 [Patent Document 5] Patent No. 6490284 Summary of the Invention [Problem to be solved by the invention]

[0013] To transfer video signals from a network camera in real time, the Real-time Transport Protocol (RTP), based on the User Datagram Protocol (UDP), is used, which transfers data without the sending node confirming receipt by the receiving node. However, RTP communication does not resend data if it does not reach the other party, so if a packet loss occurs on the network, in which part of the transferred data is lost, the video and audio will deteriorate. Therefore, such data cannot be used as is as material for video content to be viewed by viewers. On the other hand, TCP / IP (Transmission Control Protocol / Internet Protocol), which is commonly used in computer networks, prioritizes data quality and is inferior in real-time performance (timing is off), so there was a problem that video data transferred via TCP / IP was not suitable as live view video for switching or camerawork control. The aforementioned HTTP streaming enables high-quality, noise-free video and audio delivery, but as switching is not reflected in the data generated and output by each network camera or video source, data processing is required to deliver HTTP streaming that reflects switching.

[0014] However, none of the prior art, including the techniques disclosed in the above documents, has resolved these problems. The present invention has been made to solve the above-mentioned problems, and aims to provide an information processing device that is suitable for switching between images from multiple image sources, including network cameras, and delivering them via HTTP streaming. [Means for solving the problem]

[0015] The information processing device of the present invention is an information processing device that performs information processing to distribute video content using videos from multiple video sources including network cameras as material, and has a video source thumbnail display means that generates and displays a video source thumbnail based on a video signal transferred by a transfer method in which the node transmitting from the video source transfers data without confirming receipt by the receiving node, a switching means that selects and switches one of the video sources at the user's instruction, a playlist generation means that generates a playlist based on information regarding which video source was switched by the switching means and at what timing, and information processing means that distributes the video content by a transfer method that continues to retransmit the same data (packets) until it confirms a receipt notification from the receiving node. Effect of the Invention

[0016] According to the present invention, it is possible to realize an information processing device suitable for performing HTTP streaming delivery that reflects switching. [Brief description of the drawings]

[0017] [Figure 1] 1 is a block diagram showing a configuration of a main part of an information processing device according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a diagram showing an example of a mode in which an information processing device according to an embodiment is used; [Diagram 3] 2 is a block diagram showing a functional module configuration of the information processing device according to the present embodiment. FIG. [Figure 4] FIG. 2 is a block diagram showing a functional module configuration of the switcher terminal. [Diagram 5] 11A and 11B are diagrams illustrating an example of a web application screen that the information processing device according to the embodiment displays on a touch panel display of a switcher terminal. [Figure 6] 11 is a diagram showing an example of a camera operation panel that the information processing device according to the embodiment displays on a touch panel display of a switcher terminal. FIG. [Figure 7] 5 is a flowchart showing a process flow in which the information processing apparatus according to the present embodiment generates and outputs sequence data. [Figure 8] 10 is a flowchart showing an example of information processing in which the information processing device according to the present embodiment generates and distributes a playlist based on information regarding which video source was switched at what timing. [Figure 9] FIG. 13 is a diagram showing an example of a programmed camerawork panel that is displayed in response to pressing the programmed camerawork button 523. [Figure 10] 13 is a flowchart showing a process flow relating to a programmed camerawork function. [Figure 11] FIG. 2 is a diagram for explaining the "size" of a shot. [Figure 12] 5A to 5C are diagrams illustrating an example of sequence data output by the information processing device according to the embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a playlist. [Figure 14] FIG. 2 is a diagram showing an example of EDL data. [Figure 15] 1 is a block diagram showing a configuration of a main part of a network camera that can be connected to an information processing device according to an embodiment of the present invention. [Figure 16] 2 is a block diagram showing a configuration of functional modules relating to content reception and playback of a viewer terminal connectable to the information processing device according to the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Throughout this specification, "switching" refers to switching the video source (video signal from the video source), and includes a "cut switch", which is an instantaneous switch, and a "transition", which is a switch that takes a certain amount of time.

[0019] In addition, "PTZ" is the collective term for "pan," which moves the camera's angle of view horizontally, "tilt," which moves the camera's angle of view vertically, and "zoom," which enlarges and reduces the angle of view. Additionally, switching (cuts and transitions) and PTZ control (pan, tilt, and zoom control) are collectively called "camera work."

[0020] Additionally, a unit of footage shot continuously from start to finish with one camera, with no starts or stops in between, is called a "shot." Additionally, a single sequence of footage made up of switching and combining shots or similar units of footage is called a "scene." Also, a single recording of a scene with the same structure is called a "take."

[0021] Furthermore, video productions that are made available for viewing by viewers are referred to as "programs," "program content," "video content," etc. Moreover, the number indicating the order in which the scenes appear in the "video content" is called a "scene number." Also, the video of the video source being switched is called "PGM". Also, the video of the video source that is switched (before the switch) is called "PST".

[0022] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0023] FIG. 1 is a block diagram showing a configuration of a main part of an information processing device 100 according to an embodiment of the present invention. The information processing device 100 includes a CPU 110 , a RAM 120 , a ROM 130 , a display 160 , an operation unit 170 , and a network I / F 190 , which are interconnected via an internal bus 180 .

[0024] Reference numeral 110 denotes a CPU, which not only controls the operation of the information processing device 100 itself (the operation of each unit for performing its function), but also generates signals for controlling the operation of a video source and the like, and processes various types of data.

[0025] The RAM 120 is a writable high-speed storage device into which the OS, various programs, and various data are loaded, and is also used as a working area for the OS and various programs.

[0026] The ROM 130 is a non-volatile read-only storage device, and is used as a permanent storage area for the BIOS, various basic programs, and various basic data.

[0027] The display 160 is a display device such as an LCD (liquid crystal display panel).

[0028] The operation unit 170 is a keyboard, a mouse, a joystick, a remote controller, a touch panel (including a touch display), or the like.

[0029] With a touch display, a user can operate the device and execute various functions by using touch gestures such as tapping (touching down and touching up) or flicking (touching, moving quickly and then touching up) on GUI (Graphical User Interface) components such as icons with their fingers or a stylus pen.

[0030] The network I / F 190 is an I / F (interface) for connecting to a network, and enables connection to a video source such as a network camera 1500, a viewer terminal 1600, and a switcher terminal 400 via the network. The network includes, for example, the Internet, a wired LAN, a wireless LAN, a mobile phone network, and the like.

[0031] Fig. 2 is a diagram showing the overall configuration of a content output system including an information processing device according to an embodiment of the present invention. In Fig. 2, reference numeral 100 denotes an information processing device that performs content output (production or distribution), distributes video content to a viewer terminal 1600 via a network, and provides a content production function by switching to a switcher terminal 400. The viewer terminal 1600 and the switcher terminal 400 are information terminals such as personal computers (hereinafter referred to as "PCs"), tablets, and smartphones.

[0032] In the example shown in the figure, an "event" (210) such as a ceremony, sport, game, musical performance, scientific experiment, news report, social information, meeting, drama, etc. is shot from different directions (angles of view) by multiple cameras 1500-1, 1500-2, . . .

[0033] The switcher terminal 400 has, for example, a touch panel display in which a display and a touch panel are integrated, and accepts GUI operations by a user (content creator). The operation screen (GUI) displayed on this touch panel display is displayed as a WEB application on a WEB browser started on the switcher terminal 400 by the WEB server function of the information processing device 100. A video source such as a network camera 1500 is connected to the switcher terminal 400 via a network, and device control such as data access and pan / tilt is possible after authentication using the device's IP address, user name, and password. The content creator, while progressing the event with cues (signals), etc., creates video content by switching the video sources based on the video signals (real-time live view) transferred via UDP from the video sources to the switcher terminal 400. At this time, sequence data is generated that records which video sources were switched at what timing, and is transferred and stored in the RAM 120 of the information processing device 100.

[0034] Meanwhile, video data (video clips) from video sources such as cameras 1500-1, 1500-2, . . . are first stored (including cache or buffering) in storage means (storage media such as RAM or SD card) that the video sources have. When the video source is switched, the stored video data (video clip) is segmented into segment data (TS files), which are then transferred and stored in a specific storage area (for example, a folder on online storage). If necessary, the storage means (or the control means that manages it) may be equipped with a functional module for data segmentation, transfer, storage, etc. Alternatively, if the video source itself has HTTP streaming functionality, segment data (TS files) are generated and stored within each video source. Then, time information such as time codes added as metadata to the segment data (TS files) are referenced and compared with the timing at which the video source was switched, and the segment data (TS files) related to the switching timing is transferred and saved in a specific storage area (for example, a folder on online storage).

[0035] Next, the information processing device 100 according to an embodiment of the present invention generates a playlist (an index file such as an M3U file or an MPD file) that contains the file path and duration of the segment data relating to the timing at which the video source was switched, and distributes it to the viewer terminal 1600 (registers the playlist in a distribution directory). The viewer terminal 1600 references a playlist (M3U file, MPD file, etc.) and requests segment data, whereby streaming delivery of video content obtained by switching between multiple video sources is performed.

[0036] Note that segment data generated from the video of each video source may be recorded and stored in the video source itself or in a recording device (storage media) connected to it without going through a computer network, and the data may be distributed (acquired) from there to the viewer terminal 1600. In this case, the latency (delay) until the segment data can be acquired by the viewer terminal 1600 etc. is improved (delay is reduced).

[0037] As mentioned above, in UDP transfer, real-time performance takes priority and data quality decreases (noise occurs), so it is not appropriate to distribute UDP-transferred video signals as they are for viewing by viewers or to use them as material for creating video content. Therefore, the switcher terminal 400 performs switching while allowing the progress of the event to be confirmed in real time using the video signals (live view) transferred from each video source via UDP, and generates and outputs information on which video data (video source) was switched at what timing as "sequence data". This sequence data includes the ID (identifier) ​​of the video data (video clip) sent from the video source and is associated with the video data. It also has information on which video data (video source) was switched at what timing based on the time codes of the in-point (start time of switching insertion) and out-point (end time of insertion). In other words, it is possible to refer to the video data portion used as a shot in the scene of the video content. The user (content creator) can edit the video data based on the above sequence data without any loss in quality on a video editing terminal 240 connected to the recording medium 230, and add appropriate musical effects, subtitles, visual effects, etc. to complete the video content to be viewed by the audience.

[0038] Furthermore, the information processing device 100 according to this embodiment generates a playlist that describes the file paths to the segment data of the video at the timing when the video source is switched. Since TCP is used for the transfer and distribution of the segment data, data quality is ensured, and high-quality video and audio without noise is distributed to the viewer terminal 1600. In other words, the information processing device 100 according to this embodiment switches the video source based on live view video transferred via UDP from multiple video sources including a network camera, generates a playlist for content distribution based on information on which video source was switched at what timing, and based on the generated playlist, the video data of the switched video source is acquired by the viewer terminal 1600 by TCP transfer.

[0039] With the above-described configuration, the information processing device 100 according to this embodiment can switch video sources in near real time in line with the progress of an "event" such as a ceremony such as a wedding or funeral, or a drama, and can deliver video content made from video data with little degradation in quality due to network transmission, for viewing by viewers. In recent years, WebRTC (Web Real-Time Communication) has emerged as a technology for real-time video communication using UDP in a web browser, and it is also possible to use WebRTC as a method of transferring live view video from the video source for switching. In addition, when the web browser cannot display the UDP video stream, it is also possible to send the ONVIF-specified snapshot acquisition command to the network camera to continuously acquire still images (JPEG images) and perform switching based on these. Even if still images (JPEG images) are continuous, the frame rate (the number of frames displayed per unit time) is insufficient and so they cannot be viewed as is by the viewer. However, based on information regarding which video source was switched at what timing when switching was performed and on the video data output from the video source, video content without noise or reduced frame rate can be output (produced or distributed) and made available for viewing by the viewer. In other words, even when the web browser does not support video streams via UDP, it is possible to realize an information processing means for displaying a switching means on the web browser, allowing the user to switch video sources and delivering video content.

[0040] 3 is a block diagram showing a functional module configuration of the information processing device 100 according to this embodiment. The information processing device 100 includes a storage unit 301, a communication unit 302, a segment data generation unit 303, a playlist generation unit 304, a playlist distribution unit 305, a segment data distribution unit 306, a switcher unit 307, and a system clock 308. 3 are realized by information processing by software stored in the device using various hardware resources. For example, the switcher means 307 is realized by the CPU 110 processing information by software stored in the ROM 130 of the information processing device 100, and displaying a touch GUI of a web application on the touch panel display of the switcher terminal 400.

[0041] The storage means 301 stores video content in the form of a combination of video clip files output from a video source and sequence data recording the switching timing thereof, or in the form of segment data obtained by subdividing the video clip files. In addition, the IDs (identifiers), passwords, etc. of users of the switcher terminal (users of the web application), content viewers, and advertisers are stored as a producer database, a viewer database, an advertiser database, etc., respectively.

[0042] The communication means 302 transmits and receives various data and commands to and from the switcher terminal 400, the viewer terminal 1600, etc. For example, it communicates with the web browser of the switcher terminal 400 to display a web application. It also streams content in response to a request from the web browser of the viewer terminal 1600.

[0043] The segment data generating means 303 generates segment data (TS files) by dividing the video data output from the video source into very short time segments (for example, 10 seconds) in accordance with a standard format such as "MPEG-2 TS." At this time, the ID of the video source, time data of the time the video data was shot, file paths to the generated segments, etc. are stored in the storage means 301 as segment data information.

[0044] The playlist generating means 304 generates a playlist that lists the file paths and time lengths of the segment data related to the switching timing of the switched video source, from among the segment data generated by the segment data generating means 303.

[0045] The playlist distribution means 305 distributes a playlist to the viewer terminal 1600 in response to a request from the viewer terminal 1600. Specifically, the playlist is registered and saved in a distribution directory (folder).

[0046] The segment data distribution means 306 distributes the segment data (TS files) to the viewer terminal 1600. Specifically, the segment data (TS files) are registered and saved in a specific directory (folder), and the file path is recorded in a playlist.

[0047] As the event progresses, the switcher means 307, in response to instructions from the content creator user, selects and switches one of the video sources connected to the switcher terminal 400 as a constituent material for the video content of the event at that timing. The system clock 308 keeps time in synchronization with a time server.

[0048] FIG. 15 is a block diagram showing the configuration of the main parts of a network camera 1500 that can be connected to the information processing device 100 according to this embodiment. The network camera 1500 includes a CPU 1510, a RAM 1520, a ROM 1530, an imaging unit 1540, a memory I / F 1550, a network I / F 1560, a PTZ control unit 1570, and an external I / F 1580, which are interconnected via an internal bus 1590.

[0049] A CPU 1510 controls the operation of the network camera 1500 . The RAM 1520 is a writable high-speed storage device into which the OS, various programs, and various data are loaded, and is also used as a working area for the OS and various programs. The ROM 1530 is a non-volatile read-only storage device, and is used as a permanent storage area for the BIOS, various basic programs, and various basic data. An identification ID (such as an IP address) for identifying the network camera 1500 is held in the RAM 1520 or the ROM 1530 . The imaging unit 1540 photoelectrically converts an optical image using an imaging sensor such as a CCD or CMOS, and transfers the image to the CPU. The memory I / F 1550 is connected to recording media such as an SD card or HDD, and performs writing and reading of video signals and various data. The network I / F 1560 serves as an I / F for communicating with the information processing device 100 and various clients via a network in accordance with protocols such as the Internet Protocol and ONVIF. ONVIF is a protocol established by the standardization organization Open Network Video Interface Forum that implements commands for instructing device control and the start of distribution of images and metadata. The PTZ control unit 1570 obtains the pan / tilt / zoom state of a PTZ camera having a pan / tilt / zoom mechanism, and performs PTZ control according to operational instructions. Accessories such as a microphone and a viewfinder are connected to the external I / F 1580. They do not necessarily have to be directly attached to the camera body, but can be connected via a terminal via a cable, or can be connected wirelessly. When sound is picked up by the microphone, it is recorded as an audio track integrated with the video signal.

[0050] FIG. 16 is a block diagram showing the configuration of functional modules related to content reception and playback of a viewer terminal 1600 connectable to the information processing device 100 according to this embodiment. The viewer terminal 1600 comprises a communication means 1601 , a playlist acquisition means 1602 , a segment data acquisition means 1603 , a buffer means 1604 , and a playback means 1605 .

[0051] The communication unit 1601 communicates with the information processing device 100 using a protocol such as HTTP. The playlist acquisition means 1602 makes a playlist acquisition request to the information processing device 100 and acquires a playlist from the information processing device 100. Then, before playback of all segment data described in the acquired playlist is completed, the playlist acquisition request is resent and the playlist is reacquired (reloaded). If the acquired playlist indicates that it is the last playlist, acquisition of the playlist is terminated. The segment data acquisition means 1603 acquires the segment data in the order in which they are listed in the acquired playlist, that is, transmits an acquisition request to a URL (Uniform Resource Locator) indicating the file path of the TS file listed in the playlist. The buffer means 1604 buffers (temporarily holds) the acquired segment data for a certain period of time. The playback means 1605 reads out and plays back the segment data buffered in the buffer means 1604 in the order listed in the playlist.

[0052] FIG. 4 is a block diagram showing the functional module configuration of the switcher terminal 400. As shown in FIG. In FIG. 4, reference numeral 401 denotes a video source thumbnail display means that generates thumbnail images from a video signal (live view) of a video source connected to the information processing device 100 and displays them on a touch panel display of the switcher terminal 400 .

[0053] A main menu display unit 402 displays a main menu on the switcher terminal 400 . Reference numeral 403 denotes a volume level display means, which displays on the touch panel display of the switcher terminal 400 the volume level of the audio signal of a video source (hereinafter referred to as “PGM”) selected (switched) by the user from among the video sources connected to the information processing device 100. Reference numeral 404 denotes an “ON AIR” indicator display means, which displays on the touch panel display of the switcher terminal 400 that a start has been instructed by the user and sequence data, etc., consisting of information such as the video source being switched and the time of switching, is being output. Reference numeral 405 denotes an output start / end control means, which controls the start / end of output of sequence data and the like.

[0054] Reference numeral 406 denotes a video monitor means, which displays the video signal (PGM) of the video source selected (switched) by the user on a monitor section 506 on the touch panel display of the switcher terminal 400 . Reference numeral 407 denotes a video source search / add / delete means, which searches for, adds, and deletes video sources connectable to the information processing device 100 . Reference numeral 408 denotes a switching means which selects (switches) one of the video sources connected to the information processing device 100 as a PGM in response to a user instruction, and displays the video signal on a monitor section 506 on the touch panel display of the switcher terminal 400 . Reference numeral 409 denotes a camera control means, which generates a camera control command based on a user operation and transmits it to the target camera. Reference numeral 410 denotes a transition means, which controls switching from a video source selected as PGM to another video source over a certain period of time. Reference numeral 411 denotes a property means for writing / reading properties to / from a sequence (timeline) of a video source. Reference numeral 412 denotes a device setting means for setting and adjusting the display text size, screen brightness, volume, WiFi connection, etc., on the switcher terminal 400 touch panel display.

[0055] 413 is the system clock, which keeps time synchronized with a time server. Reference numeral 414 denotes a data output means, which outputs the generated sequence data in a data format such as SMIL (Synchronized Multimedia Integration Language), CSV (Comma Separated Value), EDL (Edit Decision List), or AAF (Advanced Authoring Format). 415 is a programmed camerawork means, which programs a series of camerawork (switching and PTZ control) for each scene. Reference numeral 416 denotes a camerawork execution means, which sequentially executes the camerawork programmed by the programmed camerawork means 415 based on user instructions.

[0056] Network cameras include those capable of panning, tilting, and zooming. The information processing device 100 according to this embodiment has a PTZ control means for controlling the panning, tilting, and zooming of such network cameras. By adopting such a configuration, it is possible to control the pan / tilt / zoom of the network camera in near real time in line with the progress of an "event" such as a wedding, funeral, drama, etc., and create video content using video data with little degradation in quality due to network transmission, and make it available to viewers. In addition, in this embodiment, the switching means and the PTZ control means are integrally provided. This configuration makes it possible to "direct" video content by linking switching and PTZ control along with the progression of "events" such as ceremonies such as weddings and funerals, dramas, etc. For example, in video content created by filming a wedding, the camera may zoom in on a close-up of the bride as she reads her "letter of thanks to her parents," and then switch to a camera showing her parents the moment the reading of the letter finishes.

[0057] FIG. 5 is a diagram showing an example of a web application screen that the information processing device 100 according to the embodiment of the present invention displays on the touch panel display of the switcher terminal 400. 501 is the GUI of the switcher application. Reference numeral 502 denotes a menu button for calling up the main menu. 503 is an audio meter that indicates the volume level. Reference numeral 504 denotes an "ON AIR" indicator that displays whether the device is in a recording (sequence data recording) or broadcast (streaming) state. A start button 505 instructs the start of "ON AIR" (the start of recording or broadcasting). When the start button 505 is pressed and recording or broadcasting begins, the start button 505 is replaced with a stop button 505-2. Reference numeral 506 denotes a monitor section, which is an area for displaying the video (PGM) of the video source being selected (switched).

[0058] Reference numeral 507 denotes a video source section, which is an area for displaying thumbnails of switchable video sources. Video source thumbnails 5071, 5072, ... are displayed in the video source section 507. The video source thumbnails are thumbnail icons indicating a plurality of video sources connected to the information processing device 100 via a network, and display the video output from each video source as thumbnails. In the example of FIG. 2, cameras 1500-1, 1500-2, ... are video sources, and the video captured by these cameras is displayed in real time as video source thumbnails. When no video is being captured, the last frame up to that point is displayed in a still state, or a so-called "black (pure black screen)" is displayed. In the example shown in Fig. 5, images captured by three PTZ (pan-tilt-zoom) cameras and one video image are displayed. The video image here refers to video data that has been captured in advance and stored in a recording medium such as the RAM 120 of the information processing device 100 or online storage. Such video images can also be used as material for video content. Reference numeral 508 denotes a plus button, which is a button for searching for and adding a video source to be displayed in the video source section 507 . Reference numeral 509 denotes a camera control button for controlling each video source. When the user presses the camera control button 509, a camera operation panel 600 is displayed. A camera operation panel 600 is displayed for each camera. Also, the panel can be moved to a different position as a so-called floating window. Reference numeral 510 denotes a switching type button for changing the switching type, and is composed of a cut selection button 510-1 for selecting a "cut" which is an instantaneous switching, and a transition selection button 510-2 for selecting a "transition" which is a gradual switching accompanied by an effect. The cut selection button 510-1 and the transition selection button 510-2 are so-called toggle buttons which are mutually exclusive (alternative) to each other.

[0059] Reference numeral 511 denotes a rating button that enables a user (content creator) to rate a recorded take. 512 is a timeline area that displays a so-called "filmstrip" (a thumbnail sequence of the shots that make up a scene). Reference numeral 513 denotes a comment area in the timeline area, 514 a time code scale, and 515 a video source thumbnail display area that displays (thumbnails of) the video source that was switched at the timing of that time code in the timeline area. The length of each video source thumbnail along the time code scale 514 corresponds to the duration of that "shot." Comment text entered by the user is displayed in the comment area 513. Reference numeral 516 denotes a play button, and when the play button 516 is pressed, the recorded video sequence (recorded in RAM 120 by connecting live views from each video source) is played back from the timing indicated by a sequence marker 517 on the time code scale 514 and displayed on the monitor unit 506. When the play button 516 is pressed, it is replaced with a play stop button 516-2. Also, if there is no playable video sequence (if the video sequence has not been recorded), the play button 516 is disabled (grayed out). Reference numeral 518 denotes a property button which displays a property panel, enabling a scene title, take number, etc. to be added or changed for the video sequence timeline-displayed in the timeline area 512. Also, from the property panel, scenes and takes that have been created up to that point can be called up and played back, etc. 519 is a magnification slider that adjusts the magnification at which the timeline is displayed. Reference numeral 520 denotes a fader, which is a switching slider for performing switching operations during transitions. Reference numeral 521 denotes a scene title combo box for displaying scene titles. Reference numeral 522 denotes a take number text box for displaying the take number. Numeral 523 denotes a programmed camera work button that displays a programmed camera work panel for performing the programmed camera work function. The programmed camera work function will be described later.

[0060] Since the display of the switcher terminal 400 is configured as a touch panel display integrated with a touch panel, the user can execute various functions by using a finger, a stylus pen, or the like to perform touch gestures such as tapping (touching down and touching up) or flicking (moving quickly while touching and touching up) on GUI components such as icons. For example, by tapping thumbnails 5071, 5072, ... in the video source section 507, the video source corresponding to that thumbnail can be selected (switched) as the PGM. At this time, a red frame is added to that thumbnail, indicating that the corresponding video source has been selected. Also, the video of that video source is displayed on the monitor section 506. Also, the ID of that video source (or the video clip generated from that video source) and the time of tapping (time code) are recorded as sequence data. When the user taps start button 505, recording of sequence data and recording of a video sequence (a sequence in which live views of switched video sources are connected together) are started. When the user touches down on another thumbnail while still touching one of the thumbnails in the video source section 507, the video source corresponding to the later touched thumbnail is set as the so-called preset video (hereinafter referred to as "PST") to be switched to next. Then, when the previously touched thumbnail is touched up while the later touched thumbnail is still being touched, the transition means 410 gradually changes the PGM video (video of the video source of the previously touched thumbnail) displayed on the monitor section 506 to the video of the video source set as the PST (video of the video source of the later touched thumbnail) with an effect (transition effect). That is, in this embodiment, it is possible to instruct a transition by a touch gesture on the touch display means. With this configuration, the user can operate "cut (instantaneous switching)" and "transition" by the difference between "one-point touch" and "two-point touch", and the operation method is easy to understand and convenient.

[0061] The user can also perform a transition by operating the fader 520. That is, when the user operates the fader 520, the transition means 410 gradually switches the PGM video displayed on the monitor unit 506 to the video of the video source set as PST according to the amount of sliding of the fader 520. Such transition control by fader operation is in line with the operability of conventional switcher devices, and is convenient because it allows the user to perform a transition at the timing desired. When switching (cut and transition) of a video source is executed, the ID of the video source (or video clip) and the switching start and end times (time codes) are recorded as sequence data. Reference numeral 524 denotes a transition effect pull-down for selecting a transition effect. The transition effects include "dissolve," "wipe," "slide," "squeeze," "iris," and "zoom." The transition effect pull-down 521 is enabled when the transition selection button 510-2 is selected (it is grayed out when the transition selection button 510-2 is not selected). A duration time up / down button 525 sets the duration time, which is the transition switching time. The duration time up / down button 522 is enabled when the transition selection button 510-2 is selected (it is grayed out when the transition selection button 510-2 is not selected).

[0062] FIG. 6 is a diagram showing an example of a camera operation panel that the information processing device according to the present embodiment displays on the touch panel display of the switcher terminal 400. As shown in FIG. The camera operation panel 600 has a video display section 601, a “Subject tracking” check box 602, a “Tracking even during PGM” radio button 603a, a “Do not track during PGM” radio button 603b, a “Size setting tracking” check box 604, a “Size setting” pull-down 605, a “Adjust continuity” check box 606, a close button 607, a subject designation frame 608, a pan tilt controller 609, a zoom controller 610, a focus controller 611, and a setting menu button 612. The video display section 601 is an area that displays the video captured by the corresponding camera. The pan-tilt controller 609 is a GUI that imitates a joystick for controlling the pan-tilt of a corresponding camera. The zoom controller 610 is a slider for controlling the zoom of the corresponding camera.

[0063] The user can also perform PTZ control of the corresponding camera by performing a pinch-in or pinch-out operation on the video display unit 601. That is, by performing a touch operation (pinch-in) of simultaneously touching down two points on the video display unit 601 to bring the touch positions closer to each other, the user can perform PTZ control of the corresponding camera so that the angle of view (angle) corresponding to the range displayed on the entire video display unit 601 at the time of touch-down is zoomed out and panned and tilted toward the midpoint position between the two touch positions (see FIG. 6(b)). Also, by performing a touch operation (pinch-out) of simultaneously touching down two points on the video display unit 601 to move the touch positions away from each other, the user can perform PTZ control of zoom-in and panned and tilt so that the range near the midpoint position between the two touch-down positions is expanded to the range displayed on the entire video display unit 601 (see FIG. 6(c)). By using such an operation method, the user can conveniently perform pan-tilt and zoom operations in an integrated manner using touch gestures. When such an operation is performed, an ONVIF command such as the following is sent to the network camera. AreaZoom=x1-x0,y1-y0,w1-w0,h1-h0 Here, x1, y1 are the coordinates of the midpoint of the two-point touch after the pinch operation, x0, y0 are the coordinates of the midpoint of the two-point touch before the pinch operation, w1, h1 are the width and height between the touch points after the pinch operation, and w0, h0 are the width and height between the touch points before the pinch operation. However, the width and height of the entire image display unit 601 are 200 and 200, respectively.

[0064] Generally, the so-called EASE IN / EASE OUT is considered to be a smooth and suitable PTZ movement, which is "starting slowly, gradually accelerating, and then decelerating further in the middle before stopping." Therefore, when PTZ control is performed by the above pinch operation, a speed control command may be automatically sent to make the PTZ movement EASE IN / EASE OUT within the duration of the PTZ movement. Furthermore, when pan-tilt and zoom are performed in an integrated manner, a subject that once entered the angle of view due to pan-tilt may be cut off from the angle of view again (no longer included in the angle of view) by zooming in, or a subject that once entered the angle of view due to zooming out may be cut off from the angle of view again by pan-tilt, which is generally considered undesirable and described as "flickering." Therefore, the change in the optical axis angle of pan-tilt and the change in focal length due to zooming may be calculated and controlled so that such a subject does not enter or leave the angle of view.

[0065] By the way, when a user controls the PTZ, it is convenient to know how much room there is for PTZ adjustment for the angle of view being shot at that time. Therefore, when the limit at which zoom or pan / tilt will stop if further operation is approached, the display of each side of the display frame of the video display unit 601 is made different from when such a limit is not approached, so that the user can be visually notified that there is little room for PTZ adjustment. For example, the frame surrounding the video display unit 601 is displayed with an unclear boundary so that the edge of the inner image is blurred, and when the image approaches the wide-angle end at which zooming out is not possible any more, or the telephoto end at which zooming in is not possible any more, the degree of blurring is reduced so that the boundary line indicating the frame is clearly displayed. Similarly, when the limit at which panning or tilting is possible is approached, the boundary line indicating the side of the panning direction or tilting direction is clearly displayed. By displaying in this manner, the user can appropriately control the PTZ while taking into consideration the room for PTZ. Note that the notification method using the display unit frame described above is one method for intuitively notifying the user of the PTZ limit, and other notification methods, such as displaying text such as "You are approaching the limit at which you can zoom out," may also be used. The focus controller 604 is a slider for controlling the focus of the corresponding camera. Depending on the capabilities of the corresponding camera, it can also be set to autofocus mode. The setting menu button 612 is a button for displaying a menu for setting the white balance, iris, frame rate, etc. of the corresponding camera. Note that auto white balance, auto iris, etc. can also be set depending on the performance of the corresponding camera.

[0066] The "subject tracking" checkbox 602 is a checkbox for setting automatic PTZ tracking of a specific subject. The technology of the automatic tracking function, which controls pan and tilt so that a moving object detected on the screen is kept captured near the center of the screen, has already been put to practical use in the field of surveillance cameras and the like. By selecting (checking) the "subject tracking" checkbox 602, the user can set the corresponding camera to automatically track a specific subject. When the "subject tracking" checkbox 602 is selected (checked), a subject designation frame 608 is superimposed on the video display section 601, and the user can move the frame by touch operation or the like to set the subject at the position of the frame as the subject to be tracked. The "Track during PGM" radio button 603a and the "Do not track during PGM" radio button 603b are radio buttons for setting whether or not to perform the above-mentioned automatic tracking (automatic PTZ) when the corresponding camera is switched, and are enabled when the "Subject tracking" checkbox 602 is selected (checked) (when the "Subject tracking" checkbox 602 is not selected (checked), the buttons are grayed out). The above-mentioned subject tracking is primarily aimed at keeping the subject captured on the screen, which can result in awkward and unnatural camerawork. Therefore, by not tracking during PGM, which is the image the viewer sees, it is possible to eliminate the risk of unnatural camerawork.

[0067] The “Size setting tracking” checkbox 604 is a checkbox for setting the tracking of a specific subject while maintaining the “size”, and is enabled when the “Subject tracking” checkbox 602 is selected (checked) (it is grayed out when the “Subject tracking” checkbox 602 is not selected (checked)). Here, "size" refers to how large the subject is on the screen, based on a human subject, and includes full shot (FS), full figure (FF), knee shot (KS), waist shot (WS), bust shot (BS), up shot (US), and close up (CU) (see Figure 11). The "Size setting" pull-down 605 is a pull-down for selecting and setting the above-mentioned "Size" and is enabled when the "Size setting tracking" checkbox 604 is selected (checked) (it is grayed out when the "Size setting tracking" checkbox 604 is not selected (checked)).

[0068] The "Adjust continuity" checkbox 606 is a checkbox for setting whether or not to perform automatic PTZ adjusted so that the transition (cutback) between the image of the image source that was PGM up until that point is favorable when the image of the corresponding camera is switched (changed to PGM), and is enabled by selecting (checking) the "Size setting tracking" checkbox 604 (when the "Size setting tracking" checkbox 604 is not selected (checked), it is grayed out). When an image is switched from PGM, if the "size" of the shots before and after the switch is the same (for example, if both are "bust shots" before and after the switch), the transition (switch) will look unnatural if the length from the top of the subject's head to the top of the screen is different. Also, if the "size" of the shot after the switch is tighter than the "size" of the shot before the switch (for example, if the shot is switched from "waist shot" to "bust shot"), the transition (switch) will also look unnatural if the length from the top of the subject's head to the top of the screen is not shorter before and after the switch. Also, if the "size" of the shot after the switch is looser than the "size" of the shot before the switch (for example, switching from a "bust shot" to a "waist shot"), it will look unnatural if the length from the top of the subject's head to the top of the screen is not longer before and after the switch. Therefore, when the "Adjust continuity" checkbox 606 is selected, the image of the video source set as PGM is analyzed to determine or estimate the "size" and the length from the top of the person's head to the top of the screen, and when the "size" of the PGM and the size set in the "Size setting" pull-down 605 are the same, the corresponding camera is tilted so that the length from the top of the person's head to the top of the screen is also the same. Also, when the size set in the "Size setting" pull-down 605 is tighter than the "size" of the PGM, the corresponding camera is tilted so that the length from the top of the person's head to the top of the screen is shorter than that of the PGM. Also, when the size set in the "Size setting" pull-down 605 is looser than the "size" of the PGM, the corresponding camera is tilted so that the length from the top of the person's head to the top of the screen is longer than that of the PGM. By doing so, it is possible to eliminate the possibility that the connection between the shots before and after the switch will look unnatural when the image is switched from the PGM.

[0069] The close button 607 is a button for closing (hiding) the camera operation panel 600. The subject designation frame 608 is a marker (GUI) for allowing the user to designate and set the subject within the frame as the subject to be tracked. The camera operation panel 600 can be displayed for each camera, and can be moved to a different position as a so-called floating window. As described above, in the information processing device 100 according to this embodiment, it is possible to instruct a PTZ-enabled network camera to perform PTZ by a touch gesture on the touch display means. With this configuration, the user can perform pan-tilt and zoom operations in an integrated manner. Also, it is very convenient because it is easy to intuitively understand how the angle of view is changed by PTZ control. In addition, the information processing device 100 of this embodiment detects a moving object in an image based on a video signal transferred by a transfer method in which the sending node transfers data from a PTZ-capable network camera without confirming reception by the receiving node, and performs PTZ control of the network camera so as to keep the detected moving object captured near the center of the screen. This is because such PTZ control needs to be performed based on near real-time live view video, but as already explained, the live view video transferred via UDP is not suitable for use as video content to be viewed by viewers, so by performing information processing such as TCP / IP transfer and HTTP streaming on video data other than the live view video, it is possible to deliver high-quality video content.

[0070] FIG. 11 is a diagram explaining the "size." Full Shot (FS) is the widest angle of view of the camera. A full figure (FF) is the size that includes the figure from the toes to the top of the head. Knee shot (KS) is the size that includes the length from the knee to the top of the head. Waist shot (WS) is the size from the waist to the top of the head. A bust shot (BS) is the size that includes the chest to the top of the head. A close-up (US) is a shot in which the face fills the entire screen. A close-up (CU) is a shot in which body parts such as eyes, mouth, hands, and feet fill the entire screen.

[0071] 7 is a flowchart showing a process flow in which the information processing device 100 according to this embodiment displays a video source thumbnail corresponding to a video source, changes the video displayed on the monitor unit 506 in response to a switching operation (including a fader operation) by the user, and generates and records sequence data when in a REC state. The process of this flowchart is realized by the CPU 110 of the information processing device 100 controlling each unit based on an input signal and a program. 5 on the touch panel display of the switcher terminal 400 (step S701). At this point, the information processing device 100 is ready to accept GUI operations by the user. If the user operation is pressing the plus button 508 (YES in S702), the CPU 110 searches for video sources that can be connected to the network and displays them as a list (S703). When the user selects one or more video sources and closes the list (YES in S704), a video source thumbnail of the selected video source is additionally displayed in the video source section 507 (S705). If the user's operation is pressing a camera control button of the PTZ camera (YES in S706), the CPU 110 displays the camera operation panel 600 (S707). As described above, the camera operation panel 600 has the video display section 601, the pan / tilt controller 602, the zoom controller 603, the focus controller 604, the setting menu button 605, and the close button 606. Note that, when the PTZ camera is added as a video source, the specifications such as the lens focal length at the wide-angle end and the telephoto end of the zoom and the pan / tilt movable range are acquired using the above-mentioned ONVIF protocol, etc. Also, the CPU 110 generates ONVIF commands, etc. in response to the user's operation on the camera operation panel 600, and controls the PTZ camera (S708). When the user presses the close button 606 (YES in S709), the camera operation panel 600 is closed (S710).

[0072] If the user operation is pressing the start button 505 (YES in S711), the CPU 110 transmits a command to the video source (the video source whose thumbnail is displayed in the video source section 507) connected to the information processing device 100 via the network to instruct it to start recording to online storage, recording media, or the like (S712). Also, if the scene title combo box 515 is blank (YES in S713), a scene title input panel is displayed to prompt the user to input a scene title (S714). Here, the scene title is a combination of the title (program title) of the video content as the "finished product" and the scene number, for example, "Watanabe Family Wedding Scene 1." The take number in the take number text box 516 is set to the latest take number for that scene title plus one (S715). The start button 505 is replaced with the stop button 505-2 (S716). Generation and recording of sequence data consisting of the ID of the currently switched video source (or video clip) and the switching time (time code) is started (S717). A URL (Uniform Resource Locator) can be used for the ID of the video source or video clip. A so-called filmstrip in which the names or thumbnails of the switched video sources are arranged along the time code is displayed in the timeline area 512 (S718).

[0073] When one of the video source thumbnails is tapped by the user (YES in S719), CPU 110 displays the video of the video source corresponding to that video source thumbnail on monitor unit 506 (S720). CPU 110 also displays a red frame around that thumbnail (S721) to indicate that the corresponding video source has been selected (a so-called "tally display"). CPU 110 also obtains the time (time code) from system clock 413 (S722). If the user touches down on another thumbnail while still touching one of the thumbnails in the video source section 507 (YES in S723), the video source is switched to the video source corresponding to the thumbnail touched first (S724), and the video source corresponding to the thumbnail touched later is set as PST (preset video) (S725). The thumbnail set as PST may be displayed with, for example, a blinking red frame. When the two touched thumbnails are touched up at approximately the same time and the user operates the fader 520 (YES in S726), the PGM video displayed on the monitor unit 506 is gradually switched to the PST video according to the amount of sliding of the fader 520 (S727). The effect used for the switch (transition effect) is the effect selected by the user when he or she presses the effect change button 510. The initial value for the transition effect may be set to "dissolve", as this is a commonly used transition effect. If the user does not operate fader 520 (NO in S726) and the subsequently touched thumbnail remains touched while the previously touched thumbnail is touched up (YES in S728), the PGM video is gradually switched to the PST video over the user-specified Duration Time (the time required for switching) (S729). Then, the switching of the video source is recorded as sequence data (S730). Note that the Duration Time is specified by selecting a value in the "Auto Transition Duration Time" combo box of the switching effect selection panel that is displayed when the effect change button 510 is pressed.

[0074] When the user presses the stop button 505-2 (YES in S731), a command to stop recording to the recording medium is transmitted to the video source (the video source displayed as a thumbnail on the video source section 507) connected to the information processing device 100 via the network (S732). In addition, the stop button 505-2 is replaced with the start button 505 (S733). In addition, recording of the sequence data is stopped (S734). When the user presses the menu button 502 (YES in S735), the main menu is displayed (S736). When the user selects the "Save Data" command in the main menu (YES in S737), the sequence data is saved or output to a location designated by the user (S738). Moreover, when the user selects the "Exit" command (YES in S739), the processing of this flow ends.

[0075] Fig. 12 shows an example of sequence data recorded or output by the information processing device 100 according to this embodiment. In Fig. 12, the data has a structure described in SMIL (Synchronized Multimedia Integration Language) format. SMIL is a language for synchronizing the playback of various data formats such as video, audio, and text, and is recommended for standardization by the World Wide Web Consortium (W3C). In the example of sequence data in FIG. 12, first, information not related to temporal behavior is described in an element. The meta attributes of an element are used to define properties of the data (such as program title, scene number, take number, rating, remarks, notes, etc.) and to assign values ​​to those properties. The layout attribute of an element is used to determine the display position of an object. <layout type="″text / smil-basic-layout″">< / layout> This will apply the default layout values ​​to all objects.

[0076] Information related to temporal behavior is described in the element. <seq>Below the element, objects that are played back in time sequence are described. That is, a sequence of switched video sources (video clips) is <seq>The child hierarchy of an element <video>It is written as an element. each <video>The element uses the src attribute to specify a unique ID that identifies the output file from the video source. Such an ID can be, for example, a UMID (Unique Material Identifier) ​​defined by the Society of Motion Picture and Television Engineers (SMPTE). Alternatively, the src attribute can be specified as a file path to where the output file is saved (for example, src="D:\cam1\Clip1.avi"). The clipBegin attribute specifies the start position (in point) of the switch insertion in the video clip (each output file from the video source), and the clipEnd attribute specifies the end position (out point) of the insertion. The clipBegin and clipEnd attributes can be specified in time code, for example, clipBegin="hh:mm:ss:ff". When switching between video clips involves a transition effect, for example, <transition>It is described using elements. <transition>The element's type attribute describes the type of effect, and the dur attribute describes the duration of the transition.

[0077] The data format in Fig. 12 is merely an example, and various other formats are possible, such as a Comma Separated Value (CSV) format, an Edit Decision List (EDL), an Advanced Authoring Format (AAF), or a newly defined data format. When sequence data is output in EDL (Edit Decision List) format, it is possible to read the data and edit the video using existing video editing applications, such as Adobe's Premiere Pro and Apple's Final Cut Pro.

[0078] FIG. 14 shows an example of EDL data. FIG. 14 shows an example in which program recording (recording of sequence data) starts 20 seconds after each video source starts outputting video data. In the figure, 1401 indicates the title of the sequence. Also, 1402 indicates whether the time code of the edited sequence is in the drop frame format or the non-drop frame format. Also, 1403 indicates whether the time code of the sequence of the video clip of each video source is in the drop frame format or the non-drop frame format. Additionally, 1404 represents the shot number. Also, 1405 indicates the ID of the video source. Also, 1406 represents a "2 channel audio / video" configuration. Also, 1407 indicates the switching type. Also, 1408 indicates the time code of the IN point of the video clip of the video source. Also, 1409 represents the time code of the out point of the video clip of the video source. Also, 1410 indicates the time code of the IN point of the edited video clip (video content). Also, 1411 indicates the time code of the out-point of the edited video clip (video content). When sequence data is output in a compatible data format such as EDL, it becomes possible to create video content by switching video sources, including network cameras, using existing video editing applications.

[0079] FIG. 8 is a flowchart showing an example of information processing in which the information processing device 100 according to the embodiment of the present invention generates and distributes a playlist based on information regarding which video source was switched at what timing. When a browser is started on a user terminal (PC, tablet, smartphone, etc.) and a URL (Uniform Resource Locator) of a "video content production service" provided by the information processing device 100 according to an embodiment of the present invention is input or specified, the information processing device 100 displays a homepage on (the display unit of) the user terminal (S801). It also displays a "Producer User Login" button (S802). When the user selects the “Producer User Login” button (YES in S803), authentication is performed using the user ID, password, etc. (S804), and if the authentication result is positive (YES in S805), a GUI of the switcher application such as 501 in FIG. 5 is displayed on the display of the user terminal (S806). If the user operation is to press the plus button 508 (YES in S807), network-connectable video sources are searched for and displayed as a list (S808). When the user selects one or more video sources and closes the list (YES in S809), video source thumbnails of the selected video sources are additionally displayed in the video source section 507 (S810).

[0080] If the user's operation is the pressing of a camera control button of the PTZ camera (YES in S811), the camera operation panel 600 is displayed (S812). As described above, the camera operation panel 600 has a video display section 601, a "subject tracking" check box 602, a "track even during PGM" radio button 603a, a "no tracking during PGM" radio button 603b, a "size setting tracking" check box 604, a "size setting" pull-down 605, a "continuity adjustment" radio button 606a, a "no continuity adjustment" radio button 606b, a close button 607, and a subject designation frame 608. When a touch gesture operation is performed on the video display unit 601 by the user (YES in S813), a control command corresponding to the operation is transmitted to perform PTZ camera control of the corresponding camera (S814). When subject tracking is set (YES in S815), the corresponding PTZ camera is set to tracking mode (S816). Note that the thumbnail of the corresponding PTZ camera in the video source section 507 indicates that it is in tracking mode. When the user presses the close button 607 (YES in S817), the camera operation panel 600 is closed (S818).

[0081] If the user operation is the pressing of start button 505 (YES in S819), a command is sent to the video source (video source displayed as a thumbnail in video source section 507) connected to the user terminal via the network, instructing the output of video data to a specific storage area (S820). In addition, segment data is generated by subdividing the video data output to the storage area (S821). Also, a playlist is generated that describes file paths that refer to segment data generated from the video data of the video source being switched at that time (S822), and the playlist is registered and saved in a distribution directory (folder) (S823).Also, start button 505 is replaced with stop button 505-2 (S824). Also, generation and recording of sequence data consisting of the ID of the video source (or video clip) being switched at that time and the switching time (time code) is started (S825). When the user taps one of the video source thumbnails (YES in S826), first, the switching time T1 is obtained from the system clock 308 (S827). Then, the switching type set by the switching type button 510 is determined, and if "CUT" has been selected (YES in S828), first, the video of the video source corresponding to that video source thumbnail is displayed on the monitor unit 506 (S829). A red frame is added to that thumbnail (S830) to indicate that the corresponding video source has been selected (a so-called "tally display"). Then, the imaging time added to the segment data generated from the switched video source is compared with the switching time T1, and a playlist is generated by entering file paths to segment data relating to videos imaged after time T1 (S831).

[0082] When "Transition" is selected as the switching type of the switching type button 510 (NO in S828), the PGM video displayed on the monitor unit 506 is gradually switched to the video (PST) of the video source corresponding to the tapped video source thumbnail with a user-specified effect (switching effect) for a user-specified Duration Time (switching time) (S832). A red frame is attached to the tapped video source thumbnail (S833). Segment data relating to the video (PGM) of the video source that was switched until then and the video (PST) of the newly switched video source that was shot between time T1 and Duration Time (switching time) is processed and synthesized with a user-specified effect (switching effect) to generate switching video segment data (S834), and is registered and saved in a specific directory (folder) (S835). A playlist is generated by recording the file path of the switching video segment data (S836). Then, the playlist in the distribution directory (folder) is overwritten / updated with the newly created playlist (S837). When the user presses stop button 505-2 (YES in S838), a command is sent to the connected video source (the video source whose thumbnail is displayed on video source section 507) to instruct it to stop outputting video data to a specific storage area (S839). In addition, a note is added to the playlist to indicate that it is the last playlist, and the playlist in the distribution directory (folder) is overwritten / updated (S840). In addition, stop button 505-2 is replaced with start button 505 (S841). In addition, recording of the sequence data is stopped (S842). When the user presses the menu button 502 (YES in S843), the main menu is displayed (S844). When the user selects the "Save Data" command in the main menu (YES in S845), the sequence data is saved or output to a location designated by the user (S846). Furthermore, when the user selects the "Exit" command (YES in S847), the processing of this flow ends.

[0083] FIG. 13 shows an example of a playlist in the m3u format. FIG. 13 shows an example in which switching occurs from Camera_1 to Camera_2 at 999 seconds (16 minutes and 39 seconds) from the start of the program.

[0084] The segment data for the images from Camera_1 and Camera_2 are http: / / www.contentsvision.com / sample / ts / camera_1 / http: / / www.contentsvision.com / sample / ts / camera_2 / In addition, the number of seconds in milliseconds (1 / 1000) from the point when the image capture of the video related to that data started is added as the file name of the segment data.

[0085] In the figure, 1301 indicates the file format. Also, 1302 represents the version number of the file format. Also, 1303 indicates the position of the first segment data (TS file) in the list. Also, 1304 indicates how many seconds each segment data (TS file) is divided into. Also, 1305 indicates the time length of the segment data (TS file) indicated by the file path on the next line. Also, 1306 indicates the file path (URL) to the segment data (TS file).

[0086] In this way, video content that switches between multiple video sources, including network cameras, can be distributed via HTTP streaming. TCP is used to transfer video data in HTTP streaming, allowing viewers to watch high-quality video content with reduced noise.

[0087] (Programmed camera work function) The programmed camerawork function of the information processing device according to this embodiment will be described below. Generally, ceremonies such as weddings and dramas have a prepared schedule and scenario, and the camera work (switching and PTZ) of the video content related to the ceremony or drama is performed according to the schedule and scenario. Therefore, if a series of camerawork can be programmed in advance and execution instructions can be given according to the progress of the event, it is convenient because it reduces mistakes and the burden of camerawork operation.

[0088] The programmed camerawork function of the information processing device of this embodiment is a function that allows the user to program (input) a series of camerawork in advance, and executes the programmed camerawork sequentially in response to instructions from the user. Camera work input and execution instructions are made by displaying the programmed camera work panel.

[0089] An example of the programmed camera work panel is shown in Fig. 9. The programmed camera work panel is displayed as a so-called floating window on the display of the switcher terminal 400 when the programmed camera work button 523 is pressed by the user. The programmed camerawork panel 901 has a monitor section 902 , a REC button 903 , an EDIT button 904 , a NEXT button 905 , a camerawork field 906 , an event number field 907 , a text field 908 , a BACK button 909 , a SKIP button 910 , and a pop-up 911 .

[0090] The camera work column 906 is a column showing the camera work in the order of progression. Here, the camera work refers to cut switching or transition of the video source, or PTZ control of the PTZ camera. The event number column 907 is a column showing the number of the order in which the camera work is performed. The text field 908 is a field for displaying text (such as a scenario, a progress chart, or a script) that serves as a guide for the timing of executing camerawork. The displayed text may be, for example, song lyrics, musical scale progression, beats, or bar counts. The text may also be entered by the user from an operation unit such as a keyboard, or may be read from a file, scanned from a paper manuscript, or photographed. When the file or manuscript to be read has a table or layout, only the text may be extracted and read, or the table or layout may be preserved and pasted. A so-called continuity (storyboard) may also be pasted. The monitor section 902 is an area that displays the video signal (PGM) of the video source that has been selected (switched). A REC button 903 is a button for instructing the start of recording (recording of sequence data). The EDIT button 904 is a button for switching to a mode in which the camera work column and the text column can be edited (changed, inserted, deleted, and rearranged). The NEXT button 905 is a button for executing the focused camera work and moving the focus to the next camera work. The BACK button 909 is a button for moving the focus to the previous camera work. The SKIP button 910 is a button for moving the focus to the next camera work. The popup 911 is a "speech bubble" that is displayed near the cursor position in the text and has a "BS button," a "new block button," and a "keyboard button," and is displayed for editing the text.

[0091] The user can set the programmed camera work panel 901 to EDIT mode (editable mode) and set "video source switching" and "PTZ control" in the camera work field. Also, camerawork executed in the main window (GUI 501 in FIG. 5) is stored as data and can be loaded into the programmed camerawork panel 901 and reused as programmed camerawork. In other words, for example, after switching or PTZ control is performed in the first take of a scene, that camerawork can be used as programmed camerawork in subsequent takes (reshoots), reducing the effort and mistakes involved in camerawork operations.

[0092] FIG. 10 is a flowchart showing a flow of processing related to the programmed camerawork function of the information processing device 100 according to this embodiment. When the user presses the programmed camerawork button 523, the programmed camerawork panel 901 is displayed as a so-called floating window (S1001). The information processing device 100 first determines whether there is data for camerawork performed in the main window, and if there is such data (YES in S1002), it compares it with the programmed camerawork data (camerawork data displayed on the programmed camerawork panel) (S1003). If the camerawork data executed in the main window differs from the programmed camerawork data (YES in S1004), the system displays the message "Do you want to make the executed camerawork into programmed camerawork?" and waits for the user to select YES or NO (S1005). When YES is selected by the user (YES in S1006), the executed camera work is made into a programmed camera work and is displayed in the camera work column (S1007). When the user presses the EDIT button (YES in S1008), the programmed camera work panel goes into the EDIT mode (S1009). Specifically, the REC button and NEXT button are disabled (grayed out).

[0093] In EDIT mode, when the user taps a blank spot in the camera work field (YES in S1010), a dialogue with options (radio buttons) of "Video source switching" and "PTZ" is displayed at that spot (S1011). However, if a PTZ-enabled video source is not assigned to that spot in the camera work field, the "PTZ" option is made unselectable (grayed out). When "Video source switching" is selected (YES in S1012), a dialog box is displayed having options (radio buttons) for "Cut" and "Transition", a pull-down list for selecting the video source, and a GUI similar to that of the camera control panel 601 (S1013). When the user selects "cut" and one of the video sources and the angle of view of the video source after the switch is determined (YES in S1014), the cut line, the name of that video source, and a thumbnail showing the angle of view of the video source after the switch are added and displayed in the camera work column (S1015).

[0094] When the user selects "Transition" and one of the video sources and the angle of view of the video source after the switch has been determined (YES in S1016), a cut line and the name of the selected video source and a thumbnail showing the angle of view of the video source after the switch are added and displayed in the camera work column (S1017). In addition, a band indicating the transition, a combo box indicating the switch time, and an icon indicating the type of transition effect are displayed (S1018). The combo box indicating the switch time and the icon indicating the type of transition effect have default values. The user can change the values ​​by tapping the combo box or icon. Each dialog has a cancel or close button, and when the user instructs cancel / close, the dialog returns to the previous state.

[0095] When the user selects "PTZ" (YES in S1019), a PTZ dialog having a GUI similar to that of the camera control panel 601 is displayed (S1020). When the user operates the GUI on the PTZ dialog and determines the angle of view after PTZ is performed (YES in S1021), a band indicating the cut line and PTZ, a combo box indicating the switching time, and a thumbnail indicating the angle of view after PTZ is performed are displayed in the camera work column (S1022). Note that the combo box indicating the switching time has a default value. The user can change the value by tapping the combo box.

[0096] When the user presses and holds the text field (YES in S1023), a cursor is displayed in the text (S1024). Also, a pop-up (speech bubble) is displayed (S1025). The pop-up has a BS button, a new block button, and a keyboard button. When the user presses the BS button (YES in S1026), a backspace is performed to delete the character in front of the cursor (S1027). If the object to be deleted is a "cut line" rather than a character, the text of the text block where the cursor was located is merged with the previous block, and the text blocks corresponding to the subsequent event numbers are shifted forward one by one. When the user selects the new block button (YES in S1028), the text following the cursor position in the text block is moved to a newly created block immediately following the text block, and the text blocks corresponding to subsequent event numbers are shifted forward one by one (S1029). When the user presses the keyboard button (YES in S1030), a floating keyboard is displayed, allowing text to be inserted (S1031).

[0097] When the user presses the MENU button (YES in S1032), a menu is displayed (S1033). The menu has the commands "Read text," "Save programmed camerawork data," and "Read programmed camerawork data." When the user instructs "Text Load" (YES in S1034), a file load window is displayed (S1035). The file load window has options (radio buttons) for "Overwrite" and "Insert" and "OK" and "Cancel" buttons. Note that when no file is selected in the window, the "OK" button is disabled (grayed out). When the user selects "overwrite" and loads a file (YES in S1036), all text in the text field is overwritten with the text from the loaded file (S1037). When the user selects "insert" and loads a file (YES in S1038), the text of the loaded file is inserted at the cursor position (S1039). When the user instructs "save programmed camerawork data" (YES in S1040), a data save window is displayed (S1041).

[0098] When the user instructs "Load programmed camerawork data" (YES in S1042), a data file load window is displayed (S1043). The data file load window has a check box labeled "Load executed camerawork" and "OK" and "Cancel" buttons. Note that if the check box is not checked or if no file is selected in the window, the "OK" button is disabled (grayed out). Also, if the check box is checked, file selection in the window is disabled (grayed out). When the camera work data is read (YES in S1044), the camera work column is overwritten (S1045).

[0099] When the user ends the EDIT mode (when the EDIT button that was displayed in a pressed state is tapped again) (YES in S1046), the programmed camera work panel is put into execution mode (S1047). Specifically, the REC button and NEXT button are enabled. When the user taps the NEXT button (YES in S1048), the camera movement of the focused row is performed and the focus moves to the next row (S1049). When the camera work for the last line has been executed (YES in S1050), no lines are focused on, and the NEXT button is disabled (grayed out) (S1051). When the user taps the REC button (YES in S1052), REC starts (S1053). The operation during REC is the same as when REC is instructed in step S711 in FIG. When the user taps on any of the lines in the "camera work column," "event number column," or "text column" (YES in S1054), the focus is moved to that line (S1055). When the user presses the close button or taps anywhere on the main panel (YES in S1056), the programmed camera work panel is closed (S1057). Note that REC / REC stop and the display / hide of the programmed camera work panel are independent of each other. For example, even if REC is in progress when the programmed camera work panel is closed, REC will continue.

[0100] The data used to save and load programmed camerawork is written by assigning numbers (event numbers) to video source switching and PTZ control, and relating the timing to the progress chart text and storyboard. Since it is common to sequence data in that "switching of video sources" is described, the format of sequence data may be expanded and used. An example of extending sequence data described in SMIL (Synchronized Multimedia Integration Language) format to programmed camerawork data will be described below.

[0101] First, each <video>An event_Num attribute is added to the element, which describes the order (event number) in which the camerawork is performed. <video src="″D:\cam1\Clip1.mxf″" clipBegin="″hh:mm:ss:ff" clipEnd="″hh:mm:ss:ff″" event_Num="″1″">< / video> Also, <seq>In the child hierarchy of an element, for example <onvif>Describe PTZ control as an element. <onvif CamID="″cam1″" AbsolutePanTilt="″X,Y″" AreaZoom="″x,y,w,h″" dur="″hh:mm:ss:ff″" event_Num="″2″" / > Also, corresponding to the order (event number) in which camera work is executed, describe text (such as a schedule or script) that serves as a timing guideline for executing that camera work as a <text_block> element. <text_block=″Dear guests, thank you very much for coming to today's wedding reception of ○○○○. Now, the bride and groom will enter. Please give them a big round of applause. event_Num=″1″ / >

[0102] As described above, the user can program a series of camera works in advance and sequentially execute them in the preset order by instructing the NEXT button. This can reduce the labor and operation errors of camera work operations. Also, the user can plan the timing of camera work execution in contrast to the schedule or script, further improving the operability. Also, the camera work executed in the main window can be used as programmed camera work. This reduces the labor of programming (inputting / setting) camera work. Also, an embodiment in which the above text (such as scenarios, schedules, scripts, song lyrics of music, storyboards, scores, etc.) is reused to disclose and explain the content of video content or plans and projects, and to recruit or match advertisers, collaborators, staff, etc. is also suitable. In this case, it becomes possible to build and operate platforms for planning, fundraising, production, distribution, circulation, etc. centered on video content creation. In addition, when the camerawork programmed in accordance with the above text (such as a scenario, progress chart, script, song lyrics, storyboard, and musical score) is executed and video content is produced, an embodiment in which the text is reused to notify matched advertisers, collaborators, and staff or to allow them to access all or part of the produced video content is also preferable. In this case, the notification or access to all or part of the video content (playback, viewing, confirmation, etc.) can prompt actions (actions, execution) from advertisers, collaborators, staff, etc., and the effect of accelerating the progress of projects related to video content production and distribution is achieved.

[0103] (Video content creation) As explained above, when an event is recorded by switching between multiple video sources using the information processing device 100 of this embodiment, video clips that record the video signals from the video sources are created, and sequence data is created that references the switched parts of the video clips in the order in which they were switched. This sequence data also has the nature of a so-called "playlist" that indicates the playback order of multiple video clips (segments), so by playing the video clips (segments) according to the sequence data using a video player, the video content of the recorded event can be viewed and played. For example, if the sequence data is written in SMIL format, the video content can be played using a web browser as a video player. In addition, when the sequence data is in a data format that can be read by a video editing device (video editing application), the video content of the recorded event can be written (output) as a single video file using video clips from the video source as raw data.

[0104] Since sequence data is generated for each recording (one take of one scene), when video content consists of multiple scenes, the sequence data for each scene is made to correspond to the multiple scenes (sequence data combination). For example, in the example sequence data in Figure 12 (an example of sequence data written in SMIL format), if the elements in the hierarchy below the element are combined in the order of the "scene number" written in the meta attribute of the element, they will be combined into a playlist that plays multiple scenes sequentially in scene number order.

[0105] Also, when a scene is "reshot" and there can be multiple takes for each scene, the "rating" written in the meta attribute of the element can be referenced, and the takes with the highest rating (if there are multiple, for example the one recorded later) can be linked in order of scene number to create a "playlist," which allows video content consisting of scenes that use the takes desired by the user to be viewed and played. In cases where "reshooting" is common, such as with video content for a drama, this is convenient as it makes the process of deciding which take to use for each scene more efficient and edits the video. Since text editing of sequence data (including combined data) itself produces the same results as video editing, the configuration of a video editing device (video editing application) and the resources required can be simplified and made lighter. For example, a video editing web application can be constructed by a web page that has a text editor area for editing SMIL and a monitor area for playing and displaying the SMIL. Furthermore, by having the video server refer to the file into which the information processing device 100 of this embodiment writes the sequence data, and based on that data, sequentially transmitting the video of the portion of the video clip into which the video data of the video source has been written that has been switched by the information processing device 100, it is possible to perform live streaming (with some delay) even during event recording (while sequence data is being written).

[0106] The information processing device 100 according to this embodiment switches between video sources based on live view video transferred in near real time from multiple video sources including network cameras using a transfer method in which the transmitting node transfers data without checking the reception of the receiving node, and creates or distributes video content based on information regarding which video source was switched at what timing and the video data output from the video source. With this configuration, it becomes possible to create or distribute video content using video data with little quality degradation due to network transmission, while switching video sources in near real time in line with the progress of an "event" such as a ceremony such as a wedding or funeral, or a drama. In this way, the labor required for line setup and equipment costs can be reduced, and video content production can be simplified, accelerated, and cost reduced. That is, an information processing device suitable for switching video sources including network cameras to create or distribute video content can be realized.

[0107] (Other embodiments) The present invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention.

[0108] The present invention can also be realized by supplying software (programs) that realize the functions of the above-described embodiments to a system or device via a network or various storage media, and having the computer (or CPU, MPU, etc.) of the system or device read and execute the program. Therefore, the program code itself that is installed in a computer to realize the functional processing of the present invention by the computer also realizes the present invention. In other words, the present invention also includes the computer program itself for realizing the functional processing of the present invention. In this case, as long as it has the function of the program, it may be in the form of object code, a program executed by an interpreter, a script or macro executed by an application program such as a web browser, an API (Application Programming Interface), or the like. In addition, it may be implemented by being incorporated as part of another web service such as a social networking service (SNS) using web programming techniques such as "mashup." The present invention can also be realized as a so-called WEB application that uses a WEB browser. Therefore, the WEB server indicated by the URL (Uniform Resource Locator) constitutes an embodiment of the present invention, regardless of whether it is a single piece of hardware or not. [Explanation of symbols]

[0109] 100...information processing device, 110...CPU, 120...RAM, 130...ROM, 160...display, 170...operation unit, 190...network I / F, 180...internal bus, 210...event, 400...switcher terminal, 1500...video source such as a network camera, 1600...viewer terminal, 301...storage means, 302...communication means, 303...segment data generation means, 304...playlist generation means, 305...playlist distribution means, 306...segment data distribution means, 307...switcher means, 308...system clock, 401...video source thumbnail display means, 402...media menu display means, 403...volume level display means, 404...REC indicator display means, 405...sequence recording start / end control means, 406...PGM video display means, 407...video source search / add / delete means, 408...switching means, 409...camera control means, 410...transition means, 411...property means, 412...device setting means, 413...system clock, 414...sequence data output means, 415...programmed camerawork means, 416...camerawork execution means, 501...GUI, 502...menu button, 503...audio meter, 504...ONAIR indicator, 505...start button, 506...monitor section, 507...video source section, 508...plus button, 509...camera control button, 510...switching type button, 511...rating button, 512...timeline area, 513...comment area, 514...timecode scale, 515...video source thumbnail display area, 516...play button, 517...sequence marker, 518...properties button, 519...magnification slider, 520...fader, 521...scene type Torque combo box, 522... take number text box, 523... programmed camera work button, 524... transition effect pull-down, 525... duration time up / down, 600... camera operation panel, 601... video display section, 602... "subject tracking" check box, 603a... "track even during PGM" radio button, 603b... "no tracking during PGM" radio button, 604... "size setting tracking" check box, 605... "size setting" pull-down, 606... "Adjust connection" checkbox, 607...close button, 608...subject selection frame, 609...pan-tilt controller, 610...zoom controller, 611...focus controller, 612...settings menu button, 901...programmed camera work panel, 902...monitor section, 903...REC button, 904...EDIT button, 905...NEXT button, 906...camera work field, 907...event number field, 908...text field, 909...BACK button, 910...SKIP button , 911... pop-up (speech bubble), 1510... CPU (camera), 1520... RAM (camera), 1530... ROM (camera), 1540... imaging unit (camera), 1550... memory I / F (camera), 1560... network I / F (camera), 1570... PTZ control unit (camera), 1580... external I / F (camera), 1590... internal bus (camera), 1601... viewer terminal communication means, 1602... playlist acquisition means, 1603... segment data acquisition means, 1604... buffer means, 1605... playback means< / onvif> < / seq> < / video> < / transition> < / transition> < / video> < / video> < / seq> < / seq>

Claims

1. An information processing device that performs information processing to distribute video content using video from multiple video sources including a network camera, a video source thumbnail display means for generating and displaying a video source thumbnail based on a video signal transferred by a transfer method in which a node that transmits the video signal from the video source transfers data without confirming reception of the data from a node that receives the video signal; A switching means for selectively switching one of the video sources in response to a user's instruction; a playlist generating means for generating a playlist based on information regarding which video source was switched by the switching means and at what timing; and information processing means for distributing video content by a transfer method in which the same data (packets) are continuously retransmitted until a reception notification from a receiving node is confirmed.

2. 2. The information processing apparatus according to claim 1, further comprising a touch display means, the switching means being capable of performing a transition accompanied by a switching effect, and the transition being instructable by a touch gesture on the touch display means.

3. The network camera includes a camera capable of at least panning, tilting, and zooming; 3. The information processing apparatus according to claim 1, further comprising a PTZ control means for controlling pan, tilt and zoom of the network camera.

4. 4. The information processing apparatus according to claim 3, wherein said switching means and said PTZ control means are provided integrally.

5. 5. The information processing apparatus according to claim 3, further comprising a touch display means, and wherein the pan / tilt / zoom control can be instructed by a touch gesture on the touch display means.

6. A programmed camerawork means for programming a series of camerawork in advance; 6. The information processing apparatus according to claim 1, further comprising: a camerawork execution means for sequentially executing the camerawork programmed by said programmed camerawork means based on a user instruction.

7. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 6.

8. A computer-readable recording medium storing a program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 6.

Images