Information processing device, information processing system, information processing method, and program

Abstract

To allow stream distribution to be continued while suppressing an increase in the amount of data transmission.SOLUTION: An information processing device includes a processing unit. The processing unit acquires a first stream transmitted from a transmission device. When a transmission request that requests a stream to be transmitted from a second server device to a receiving device is received regarding a first server device that stores an input stream in a storage device and the second server device that transmits the stream so as to have a delay within a second delay time that is shorter than a first delay time when the stream stored in the storage device is distributed, the processing unit relays the acquired first stream to both first server device and second server device. When the transmission request is not received, the processing unit relays the acquired first stream to the first server device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] A system is known that collects and stores video (streams) captured by a camera (imaging device) and distributes them to a receiving device. One example of such a system is a technology that switches between a high-quality mode in which the stream is encoded at high quality and a low-latency mode in which the stream is transmitted with low latency. The low-latency mode is used, for example, when operating the camera, such as turning, zooming, and focusing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-284051 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the above technology, when the camera is in low-latency mode, the delivery of high-quality streams is stopped. In other words, only one stream can be transmitted, either in high-quality mode or low-latency mode. While providing a system for high-quality mode and a system for low-latency mode would enable transmission of both streams, problems such as an increase in data transmission volume could arise due to the camera transmitting streams to two systems.

[0005] An object of the present invention is to provide an information processing device, an information processing system, an information processing method, and a program that are capable of continuing to distribute a stream while suppressing an increase in the amount of data transmission. [Means for solving the problem]

[0006] An information processing device according to an embodiment includes a processing unit. The processing unit acquires a first stream transmitted from a transmission device. When a transmission request for transmitting a stream to a receiving device is received from a first server device that stores the input stream in a storage device and a second server device that transmits the stream so as to have a delay within a second delay time that is smaller than a first delay time when the stream stored in the storage device is distributed, the processing unit relays the acquired first stream to both the first server device and the second server device. When a transmission request is not received from the second server device, the processing unit relays the acquired first stream to the first server device. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram of an information processing system according to a first embodiment. [Figure 2] 10 is a flowchart of a collection process according to the first embodiment. [Figure 3] 10 is a flowchart of a large-scale distribution process in a normal mode according to the first embodiment. [Figure 4] 10 is a flowchart of a parallel mode switching process according to the first embodiment. [Figure 5] 10 is a flowchart of a process for returning to a normal mode in the first embodiment. [Figure 6] FIG. 10 is a block diagram of an information processing system according to a second embodiment. [Figure 7] 10 is a flowchart of a parallel mode switching process according to the second embodiment. [Figure 8] 10 is a flowchart of a process for returning to a normal mode in the second embodiment. [Figure 9] FIG. 10 is a block diagram of an information processing system according to a first modified example. [Figure 10] FIG. 2 is a hardware configuration diagram of the device according to the first or second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an information processing apparatus according to the present invention will be described in detail below with reference to the accompanying drawings.

[0009] (First embodiment) The information processing system of the first embodiment includes a video server for distributing streams, a low-latency server for transmitting streams with low latency, and an information processing device that relays streams transmitted from cameras to the video server and the low-latency server. The information processing device relays the streams transmitted from cameras to both the video server and the low-latency server only while the low-latency server is being accessed. When the low-latency server is not being accessed, the information processing device relays the streams transmitted from cameras only to the video server. This makes it possible to continue distributing streams while suppressing an increase in the amount of data transmitted from cameras, for example.

[0010] For example, it may be necessary to receive a stream with low latency in order to adjust the camera angle of view, etc. In such a case, for example, a technique for switching from a high-quality mode to a low-latency mode stops the delivery of the stream at high quality. In this embodiment, the delivery of the stream at high quality can be executed in parallel with the reception of the stream at low latency.

[0011] Fig. 1 is a block diagram showing an example of the configuration of an information processing system according to the first embodiment. As shown in Fig. 1, the information processing system according to the present embodiment includes an information processing device 100, a cloud 200, a plurality of receiving devices 300a, 300b, and 300c, a receiving device 400, and a camera 500.

[0012] The multiple receiving devices 300a, 300b, and 300c have the same configuration, and therefore are simply referred to as receiving devices 300 when there is no need to distinguish between them. The number of receiving devices 300 is not limited to three, and may be two, four, or more. The number of other devices is also not limited to that shown in FIG. 1. For example, the information processing system may include multiple cameras 500 and multiple receiving devices 400.

[0013] 1, the devices constituting the information processing system (information processing device 100, cloud 200, receiving device 300, receiving device 400, camera 500) are connected to each other via a network such as the Internet. The network may be a wired network, a wireless network, or a network in which wired and wireless networks are mixed.

[0014] The camera 500 corresponds to an imaging device that captures video. The camera 500 also corresponds to a transmission device that transmits the captured video to the information processing device 100. The transmission device is not limited to the camera 500 (imaging device). The camera 500 includes an encoding unit 501 and a transmission unit 502.

[0015] The encoding unit 501 encodes the captured video. Any method may be used for encoding the video by the encoding unit 501. The transmission unit 502 transmits the encoded video (stream) to the information processing device 100.

[0016] Cloud 200 includes a video server 210 (first server device), a low-latency server 220 (second server device), and a web server 230 (third server device). Cloud 200 means that these three servers are constructed on a cloud environment. Some of the three servers may be constructed on a cloud environment different from the remaining servers. The method of constructing the three servers is not limited to constructing them on a cloud environment. For example, some or all of the three servers may be constructed as independent server devices that are not in a cloud environment.

[0017] The video server 210 includes a storage device 211, and stores the input stream in the storage device 211. For example, the video server 210 receives a stream from the information processing device 100, and stores the received stream in the storage device 211 as a file.

[0018] The file may have any data structure, for example, the following structure: Note that the file creation time and data format may be combined in any way. File creation time: One file per frame, or one file per segment Data format: NAL (Network Abstraction Layer) stream structure, or converted to MP4 or other formats

[0019] The storage device 211 can be configured from any commonly used storage medium such as a flash memory, a memory card, a RAM (Random Access Memory), an HDD (Hard Disk Drive), and an optical disk.

[0020] The low-delay server 220 is a server device that transmits streams with less delay than the video server 210. For example, the low-delay server 220 transmits the stream received from the information processing device 100 so that the delay is within a delay time T2 (second delay time) that is shorter than the delay time T1 (first delay time) when the stream stored in the storage device 211 of the video server 210 is distributed.

[0021] The web server 230 is a server that distributes the stream stored in the storage device 211 of the video server 210. For example, the web server 230 acquires the stream from the storage device 211 of the video server 210 and distributes the acquired stream to the receiving device 300.

[0022] Receiving device 300 is a device that receives a stream distributed from web server 230 and decodes and displays the received stream. For example, when receiving device 300 accesses an address for distributing the stream (e.g., a Uniform Resource Locator (URL)), it receives a viewer for displaying the stream from web server 230. Hereinafter, an address for acquiring a viewer for stream distribution may be referred to as a large-scale distribution URL. This viewer has a function for receiving and displaying the stream from web server 230. Receiving device 300 decodes and displays the distributed stream using the received viewer. Note that the viewer may be preset in receiving device 300 instead of being transmitted from web server 230.

[0023] For example, HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH) can be used to deliver streams between the web server 230 and the viewer. HLS and DASH can have large delays due to the following features: The stream is delivered as a set of files called segments, each containing a certain amount of time. To prevent playback from stalling, the viewer buffers two segments before playing the stream. The stream is sent from the web server 230 after being requested by the viewer.

[0024] The receiving device 400 is a device that receives a stream transmitted from the low-delay server 220 and decodes and displays the received stream. For example, when the receiving device 400 accesses an address (e.g., a URL) for transmitting a stream with low latency, it receives a viewer for displaying the stream (hereinafter referred to as a low-delay viewer) from the web server 230. The low-delay viewer may be configured to be received from the low-delay server 220. Alternatively, the low-delay viewer may be pre-set in the receiving device 400 instead of being transmitted from the web server 230. Hereinafter, the address for acquiring the low-delay viewer may be referred to as a low-delay URL. The low-delay viewer has a function of receiving and displaying the stream from the low-delay server 220 with low latency. The receiving device 400 decodes and displays the distributed stream using the received low-delay viewer.

[0025] For example, Web Real-Time Communication (WebRTC) can be used to transmit streams between the low-latency server 220 and the low-latency viewer. WebRTC is not suitable for large-scale distribution due to the following features, but it can reduce delays. Note that it is not suitable for large-scale distribution because the load on the distribution side increases as the number of distribution destinations increases. There is no need to wait for one segment before converting the stream to a file. - There is no need to wait two segments of the stream and buffer it on the viewer side. There is no need to wait for a request from the viewer to send the stream.

[0026] Generally, multiple systems are used for different use cases, such as a system using HLS and DASH for large-scale distribution, a system using WebRTC for low-latency transmission, etc. However, in such a configuration, it becomes necessary to transmit streams from the camera to multiple systems, which can increase the amount of data transmitted.

[0027] In contrast, in this embodiment, the information processing device 100 relays the same stream transmitted from the camera to both the web server 230 that delivers using HLS or DASH, and the low-latency server 220 that transmits using WebRTC, for example. This makes it possible to achieve both large-scale delivery and low-latency transmission.

[0028] The information processing device 100 includes a processing unit 110. The processing unit 110 includes an acquisition unit 111, a relay unit 112, and a switching unit 113. The information processing device 100 may be physically configured as a single device, or may be physically configured as a plurality of devices. For example, the information processing device 100 may be built on the same cloud environment as the cloud 200 or on a cloud environment different from the cloud 200.

[0029] The acquisition unit 111 acquires various data from an external device such as the camera 500. For example, the acquisition unit 111 acquires a stream ST1 (first stream) transmitted from the camera 500.

[0030] The relay unit 112 relays the acquired stream ST1 to the video server 210 and the low-delay server 220. For example, when a transmission request requesting transmission of a stream from the low-delay server 220 to the receiving device 400 is received, for example, by the switching unit 113, the relay unit 112 relays the stream ST1 to both the video server 210 and the low-delay server 220. The mode in which the stream ST1 is relayed to both the video server 210 and the low-delay server 220 may be referred to as a parallel mode hereinafter.

[0031] If the relay unit 112 has not received a transmission request from the low-delay server 220 to transmit a stream to the receiving device 400, the relay unit 112 relays the stream ST1 to the video server 210 but does not relay it to the low-delay server 220. The mode in which the stream ST1 is relayed to the video server 210 but not to the low-delay server 220 may be referred to as the normal mode hereinafter.

[0032] The transmission request is transmitted from the receiving device 400 to the information processing device 100, for example, via the low-latency server 220. For example, the receiving device 400 uses a low-latency viewer to transmit a stream transmission request to the low-latency server 220. The low-latency server 220 transmits the transmission request transmitted by the low-latency viewer to the information processing device 100.

[0033] The relay unit 112 can be configured to operate in the normal mode or the parallel mode in response to the mode switching by the switching unit 113 .

[0034] That is, the switching unit 113 switches the mode of relay operation by the relay unit 112 in response to a transmission request received from the low-latency server 220. For example, when a transmission request is received from the low-latency server 220, the switching unit 113 switches the mode of relay operation by the relay unit 112 from normal mode to parallel mode. When access from the low-latency viewer is completed, the switching unit 113 switches the mode of relay operation by the relay unit 112 from parallel mode to normal mode. In this way, the parallel mode can be interpreted as a temporary mode that transitions only when access is being made by a low-latency viewer.

[0035] The processing unit 110 is realized by, for example, one or more processors. For example, the processing unit 110 may be realized by causing a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) to execute a program, i.e., by software. The processing unit 110 may be realized by a processor such as a dedicated IC (Integrated Circuit), i.e., by hardware. The processing unit 110 may be realized by a combination of software and hardware. When multiple processors are used, each processor may realize one of the units included in the processing unit 110 (the acquisition unit 111, the relay unit 112, and the switching unit 113), or may realize two or more of the units.

[0036] Next, various processes performed by the information processing system of the first embodiment will be described with reference to Figs. 2 to 5. Fig. 2 is a flowchart showing an outline of the collection process in the first embodiment. The collection process is a process of collecting and storing images captured by the camera 500 in the video server 210. Note that Fig. 2 corresponds to an example in which the relay unit 112 operates in normal mode. For example, in the initial state after the information processing device 100 is started up, the relay unit 112 is set to normal mode.

[0037] The encoding unit 501 of the camera 500 encodes the video captured by the camera 500 (step S101). The transmission unit 502 transmits the encoded video (stream) to the information processing device 100 (step S102).

[0038] The acquisition unit 111 of the information processing device 100 acquires the stream transmitted from the camera 500 and passes it to the relay unit 112. The relay unit 112 relays the received stream only to the video server 210 (step S103).

[0039] The video server 210 stores the received stream as a file in the storage device 211 (step S104). For example, the video server 210 waits until it receives a stream for the time required for the file (one frame, one segment, or the like), performs format conversion as necessary, converts the stream into a file, and stores the file in the storage device 211. If the camera 500 transmits the stream in segments, the video server 210 does not need to wait for the time required for the file to be generated.

[0040] Next, an overview of the large-scale distribution process in normal mode will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an overview of the large-scale distribution process in normal mode in the first embodiment.

[0041] For example, the user of the receiving device 300 specifies a URL for large-scale distribution using a browser, etc. The receiving device 300 accesses the web server 230 in accordance with the specification of the URL for large-scale distribution (step S201).

[0042] The web server 230 transmits the viewer to the accessing receiving device 300 (step S202).

[0043] Using the transmitted viewer, the receiving device 300 transmits a request for the stream to the web server 230, for example, periodically (step S203).

[0044] The web server 230 distributes the stream requested by the viewer to the requesting receiving device 300 (step S204). Although not shown in FIG. 3, for example, the web server 230 reads the requested stream from the storage device 211 of the video server 210 and distributes the read stream to the receiving device 300.

[0045] The receiving device 300 receives and decodes the stream from the web server 230 using the viewer, and displays the decoded stream on a display device or the like provided in the receiving device 300 (step S205).

[0046] Next, an overview of the process of switching to parallel mode (hereinafter referred to as parallel mode switching process) will be explained using Fig. 4. When switching to parallel mode, the process differs from Fig. 2 in that the stream is relayed not only to the video server 210 but also to the low-latency server 220. Fig. 4 is a flowchart showing an overview of the parallel mode switching process in the first embodiment.

[0047] For example, the user of the receiving device 400 specifies a low-delay URL using a browser, etc. The receiving device 400 accesses the web server 230 in accordance with the specification of the low-delay URL (step S301).

[0048] The web server 230 transmits a low-delay viewer to the accessing receiving device 400 (step S302).

[0049] The receiving device 400 uses the transmitted low-delay viewer to access the low-delay server 220 and establish a connection with the low-delay server 220 (step S303).

[0050] The low-delay server 220 notifies the switching unit 113 of the information processing device 100 that it has been accessed by a low-delay viewer (step S304).

[0051] The switching unit 113 of the information processing device 100 instructs the relay unit 112 to switch to the parallel mode (step S305).

[0052] Note that although the process of transmitting the stream from the camera 500 to the information processing device 100 (corresponding to steps S101 and S102 in FIG. 2) is omitted in FIG. 4, this transmission process can be continuously executed regardless of the mode of the relay unit 112.

[0053] After receiving the instruction to switch to the parallel mode, the relay unit 112 relays the stream received from the transmission unit 502 of the camera 500 to both the video server 210 and the low-delay server 220 (steps S306 and S307).

[0054] The low-delay server 220 transmits the stream to the receiving device 400 (step S308). Although not shown in Fig. 4, the stream relayed to the video server 210 is subject to large-scale distribution processing similar to that shown in Fig. 3. That is, in parallel mode, large-scale distribution processing (Fig. 3) and low-delay transmission (steps S308 and S309) are executed in parallel.

[0055] The receiving device 400 receives the stream from the low-delay server 220 with low delay using the low-delay viewer, and decodes and displays the received stream (step S309).

[0056] Next, an outline of the process of returning from the parallel mode to the normal mode will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an outline of the process of returning from the parallel mode to the normal mode in the first embodiment.

[0057] For example, the user of the receiving device 400 instructs the low latency viewer to be terminated. The receiving device 400 follows the termination instruction and terminates the low latency viewer (step S401).

[0058] In response to the termination of the low delay viewer, the receiving device 400 disconnects the connection with the low delay server 220 (step S402).

[0059] If multiple low-latency viewers (receiving devices 400) can be connected, the low-latency server 220 determines whether the connections with all low-latency viewers have been disconnected (step S403). If the connections with all low-latency viewers have not been disconnected (step S403: No), the process of returning from parallel mode to normal mode is interrupted, and parallel mode is maintained.

[0060] If the connections with all low latency viewers have been disconnected (step S403: Yes), the low latency server 220 notifies the switching unit 113 of the information processing device 100 that there is no longer any access by low latency viewers (step S404).

[0061] The switching unit 113 of the information processing device 100 instructs the relay unit 112 to end the parallel mode and return to the normal mode (step S405).

[0062] After receiving the instruction to switch to the normal mode, the relay unit 112 relays the stream received from the transmission unit 502 of the camera 500 only to the video server 210 (step S406).

[0063] In this way, in the first embodiment, only while there is access from a low-latency viewer, the stream is relayed to both the video server 210 for large-scale distribution and the low-latency server 220 for low-latency transmission. This makes it possible to achieve both large-scale distribution and low-latency transmission without stopping the stream on the large-scale distribution side and without increasing the amount of data transmitted from the camera 500.

[0064] It should be noted that the relay unit 112 and the low-delay server 220 do not need to be always running, and may be started only when the mode is switched to the parallel mode. For example, the switching unit 113 may start at least one of the relay unit 112 and the low-delay server 220 when a transmission request requesting transmission of a stream to the receiving device 400 is received from the low-delay server 220.

[0065] Starting the relay unit 112 means, for example, starting execution of a program corresponding to the relay unit 112 on the information processing device 100, or starting operation of a circuit corresponding to the relay unit 112. Starting the low-latency server 220 means, for example, starting execution of a program corresponding to the low-latency server 220 on the cloud 200.

[0066] If the relay unit 112 is not activated, for example in normal mode, the transmitter 502 of the camera 500 may transmit the stream directly to the video server.

[0067] The switching unit 113 may be provided in the camera 500. For example, the camera 500 may include a switch (which may be hardware or software) for switching between the normal mode and the parallel mode. The switching unit 113 provided in the camera 500 instructs the information processing device 100 to switch between the normal mode and the parallel mode in response to the switching of the switch. The switching unit 113 may switch the mode when a predetermined image (such as a QR code (registered trademark)) is captured by the camera 500.

[0068] In this case, a transmission request requesting that the low-latency server 220 transmit a stream to the receiving device 400 is transmitted from the camera 500 to the information processing device 100. For example, when the transmission request is received from the camera 500, the relay unit 112 relays the stream ST1 to both the video server 210 and the low-latency server 220.

[0069] The switching unit 113 provided in the camera 500 may activate at least one of the relay unit 112 and the low-latency server 220 when switched to the parallel mode.

[0070] In the above description, a distinction has been made between a viewer used for large-scale distribution and a low-latency viewer, but a single viewer that includes both functions may also be used. In this case, the receiving device 300 and the receiving device 400 may be integrated. For example, a user of the receiving device 300 (receiving device 400) accesses the web server 230 using a browser or the like and receives a viewer that includes both large-scale distribution and low-latency transmission functions from the web server 230. This viewer allows the user to specify either large-scale distribution or low-latency transmission using, for example, a radio button. If large-scale distribution is specified, the viewer accesses the web server 230, receives the large-scale distribution stream, and displays it. If low-latency transmission is specified, the viewer accesses the low-latency server 220, receives the low-latency transmission stream, and displays it.

[0071] The information processing system may be divided into an information processing system SA including some of the devices and an information processing system SB including the other devices. The information processing systems SA and SB may be built, for example, on different cloud environments. An example configuration of the information processing system SA is shown below. (Configuration Example 1) Information processing system SA including information processing device 100 and web server 230 (Configuration Example 2) An information processing system SA including an information processing device 100, a video server 210, and a low-latency server 220 (Configuration Example 3) Information Processing System SA Including Information Processing Device 100 and Camera 500

[0072] As described above, in the first embodiment, when there is access to the low-latency server 220, the stream is relayed to both the video server 210 and the low-latency server 220, and when there is no access to the low-latency server 220, the stream is relayed only to the video server 210. This makes it possible to continue distributing the stream while suppressing an increase in the amount of data transmission.

[0073] (Second embodiment) In the first embodiment, the same stream is transmitted in both normal mode and parallel mode. In the second embodiment, in parallel mode, if it is determined that low latency cannot be achieved with the stream in normal mode, the encoding parameters are changed (adjusted) so that low latency can be achieved.

[0074] Fig. 6 is a block diagram showing an example of the configuration of an information processing system according to the second embodiment. As shown in Fig. 6, the information processing system according to the second embodiment includes an information processing device 100-2, a cloud 200, a plurality of receiving devices 300a, 300b, and 300c, a receiving device 400, and a camera 500-2.

[0075] The information processing device 100-2 includes a processing unit 110-2. The processing unit 110-2 includes an acquisition unit 111, a relay unit 112, and a switching unit 113-2. The camera 500-2 includes an encoding unit 501-2, a transmission unit 502, a determination unit 503-2, and an adjustment unit 504-2.

[0076] The second embodiment differs from the first embodiment in the functions of a switching unit 113-2, an encoding unit 501-2, and the fact that a camera 500-2 further includes a determination unit 503-2 and an adjustment unit 504-2. The other configurations and functions are the same as those of the information processing device 100 of the first embodiment in FIG. 1, which is a block diagram of the information processing device 100, and therefore the same reference numerals are used and the description thereof will be omitted here.

[0077] The switching unit 113-2 further has a function of notifying the determination unit 503-2 that the mode has been switched to the parallel mode when the mode has been switched to the parallel mode.

[0078] The determination unit 503-2 determines whether or not it is possible to achieve low delay in transmission of the stream captured in the normal mode from the low-latency server 220. For example, when switching to the parallel mode, the determination unit 503-2 determines whether or not the stream acquired (captured) by the camera 500 can be transmitted with a delay within the delay time T2.

[0079] When the judgment unit 503-2 determines that low delay cannot be achieved, the adjustment unit 504-2 adjusts the encoding parameters (encoding parameters) used by the encoding unit 501-2 so that low delay can be achieved, and instructs the adjusted encoding parameters to the encoding unit 501-2.

[0080] The encoding unit 501-2 performs encoding using the encoding parameters instructed by the adjustment unit 504-2.

[0081] The determination method by the determination unit 503-2 and examples of the encoding parameters will be described later.

[0082] Next, various processes performed by the information processing device 100-2 of the second embodiment will be described. The collection process and the large-scale distribution process in the normal mode are the same as those in the first embodiment (FIGS. 2 and 3), and therefore descriptions thereof will be omitted.

[0083] FIG. 7 is a flowchart showing an outline of the parallel mode switching process in the second embodiment.

[0084] Steps S501 to S505 are the same as steps S301 to S305 in the information processing apparatus 100 of the first embodiment, and therefore description thereof will be omitted.

[0085] In this embodiment, the switching unit 113-2 notifies the determination unit 503-2 of the camera 500 that the mode has been switched to the parallel mode (step S506). Note that, in order to identify the camera transmitting the specified stream, the information processing device 100-2, for example, stores correspondence information indicating the correspondence between the camera 500-2 and the stream. The correspondence information is, for example, information indicating the correspondence between the address (such as an IP address) of the camera 500-2 and the stream. Furthermore, the information processing device 100-2 stores information such as the address of the camera 500-2 in order to transmit an instruction to the camera 500-2.

[0086] The determination unit 503-2 of the camera 500-2 determines whether low latency can be achieved with the stream in normal mode (step S507). If low latency cannot be achieved (step S507: No), the adjustment unit 504-2 adjusts the encoding parameters to enable low latency and instructs the encoding unit 501-2 to perform encoding using the adjusted encoding parameters (step S508).

[0087] After the encoding parameters are adjusted, or if it is determined that low delay can be achieved (step S507: Yes), the encoding unit 501-2 encodes the subsequently captured stream using the instructed encoding parameters (step S509). The transmission unit 502 transmits the encoded stream to the information processing device 100-2 (step S510).

[0088] Steps S511 to S514 are the same as steps S306 to S309 in the information processing apparatus 100 of the first embodiment, and therefore description thereof will be omitted.

[0089] Next, an outline of the process of returning from the parallel mode to the normal mode in this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an outline of the process of returning from the parallel mode to the normal mode in the second embodiment.

[0090] Steps S601 to S605 are the same as steps S401 to S405 in the information processing apparatus 100 of the first embodiment, and therefore description thereof will be omitted.

[0091] When the parallel mode is ended, the switching unit 113-2 notifies the determination unit 503-2 of the camera 500-2 that the parallel mode has ended (step S606).

[0092] The determination unit 503-2 notifies the adjustment unit 504-2 that the parallel mode has ended. The adjustment unit 504-2 instructs the encoding unit 501-2 to change to the encoding parameters for the normal mode (step S607). Note that if the encoding parameters have not been adjusted in step S508 of FIG. 7, the adjustment unit 504-2 does not need to change to the encoding parameters for the normal mode.

[0093] The encoding unit 501-2 uses the encoding parameters instructed by the adjustment unit 504-2 to encode the streams captured thereafter (step S608).

[0094] The transmission unit 502 transmits the encoded stream to the information processing device 100-2 (step S609).

[0095] The relay unit 112 of the information processing device 100-2 relays the stream received from the transmission unit 502 only to the video server 210 (step S610).

[0096] Next, an example of the determination process by the determination unit 503-2 will be described. The determination unit 503-2 determines whether or not a transmission delay will occur, that is, whether or not a reduction in delay can be achieved, by, for example, the following determination method. (Determination method 1) The camera 500-2 and the information processing device 100-2 synchronize their time. Transmission time information is embedded in the stream transmitted from the camera 500-2, and the determination unit 503-2 calculates the delay time, which is the difference between the transmission time of the stream recorded on the camera 500-2 side and the reception time of the stream recorded on the information processing device 100-2 side. If the calculated delay time is within an allowable range, the determination unit 503-2 determines that low delay can be achieved. (Determination Method 2) A test program is installed in at least one of the camera 500-2 and the information processing device 100-2. The test program, for example, transmits test data from the camera 500-2 to the information processing device 100-2 and estimates the maximum bit rate for communication between the two. The determination unit 503-2 determines that low latency can be achieved if the stream can be transmitted within the estimated maximum bit rate.

[0097] Next, an example of the encoding parameter adjustment process performed by the adjustment unit 504-2 will be described.

[0098] The stream includes, for example, I frames, which contain the entire image frame information, and P and B frames, which contain only the difference information between the previous and next frames. Since I frames contain the entire image frame information, they have a large data size. On the other hand, since P and B frames contain only the difference information, they have a small data size.

[0099] As mentioned above, with HLS and DASH, the stream is played after two segments are buffered. Therefore, even if it takes two segments to transmit, there is no impact on display delays. On the other hand, with parallel mode, there is no restriction to wait two segments, so on slow network lines, it may take a long time to transmit an I-frame with a large data size, resulting in display delays. In other words, low latency may not be achieved.

[0100] Taking such a situation into consideration, when the determining unit 503-2 determines that a low delay cannot be achieved, the adjusting unit 504-2 adjusts the encoding parameters using the following adjustment method. (Adjustment method 1) For example, if the transmission time of an I frame is longer than a threshold (if the delay time of an I frame exceeds the allowable range), in order to reduce the transmission time of the I frame, the adjustment unit 504-2 changes the encoding parameters using an intra-refresh mechanism so that the I frame is divided and encoded. (Adjustment Method 2) For example, if the transmission time of an I frame is longer than a threshold, the adjustment unit 504-2 changes the encoding parameters so as to change the balance of the code amounts of the I frame and P frame and reduce the data size of the I frame. (Adjustment Method 3) The adjustment unit 504-2 adjusts the encoding parameters in consideration of information about the network line used by the transmission unit 502. For example, if the network line is set so as not to transmit data if the specified bit rate is exceeded, the adjustment unit 504-2 may adjust the encoding parameters and adjust the maximum data size of an I frame so that the specified bit rate is not exceeded.

[0101] The determining unit 503-2 and the adjusting unit 504-2 may be configured to repeatedly execute the determining process and the adjusting process until it is determined that low delay can be achieved.

[0102] Note that adjusting the encoding parameters to reduce delay may result in a decrease in image quality. Therefore, the adjustment unit 504-2 may adjust the encoding parameters so as to temporarily increase the amount of code in order to suppress the decrease in image quality. For example, the adjustment unit 504-2 may set a maximum bit rate to maintain image quality only while switching to the parallel mode, and, if communication is possible within the maximum bit rate, change the encoding parameters to temporarily increase the amount of code.

[0103] (Variation 1) At least one of the determining unit 503-2 and the adjusting unit 504-2 may be provided in the information processing device 100-2. Fig. 9 is a block diagram showing an example of the configuration of the information processing system of the first modification.

[0104] As shown in FIG. 9, the information processing system of the first modification includes an information processing device 100-2b, a cloud 200, a plurality of receiving devices 300a, 300b, and 300c, a receiving device 400, and a camera 500-2b.

[0105] In this modification, camera 500-2b does not include determination unit 503-2 and adjustment unit 504-2, and information processing device 100-2b is configured to include determination unit 503-2 and adjustment unit 504-2. Since the other configurations are the same as those in the second embodiment, the same reference numerals are used and the description here will be omitted.

[0106] As described above, in the second embodiment, if low latency cannot be achieved with a stream in normal mode, the encoding parameters can be changed only while there is access from a low-latency viewer. This makes it possible to achieve both large-scale distribution and low-latency transmission without increasing the amount of data transmitted from the camera.

[0107] As described above, according to the first and second embodiments, it is possible to continue the distribution of a stream while suppressing an increase in the amount of data transmission.

[0108] Next, the hardware configuration of the device (information processing device, receiving device, video server, low-latency server, web server) of the first or second embodiment will be described with reference to Fig. 10. Fig. 10 is an explanatory diagram showing an example of the hardware configuration of the device of the first or second embodiment.

[0109] The device of the first or second embodiment includes a control device such as a CPU (Central Processing Unit) 51, a storage device such as a ROM (Read Only Memory) 52 or a RAM (Random Access Memory) 53, a communication I / F 54 that connects to a network and communicates, and a bus 61 that connects each part.

[0110] The program executed by the device of the first or second embodiment is provided in advance in the ROM 52 or the like.

[0111] The program executed by the device of the first or second embodiment may be configured to be provided as a computer program product by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disk).

[0112] Furthermore, the program executed by the device of the first or second embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the device of the first or second embodiment may be provided or distributed via a network such as the Internet.

[0113] The program executed by the device of the first or second embodiment can cause a computer to function as each part of the device described above. In this computer, the CPU 51 can read the program from a computer-readable storage medium onto the main storage device and execute it.

[0114] A configuration example of the embodiment will be described below. (Configuration example 1) Acquire a first stream transmitted from a transmission device; a first server device that stores an input stream in a storage device, and a second server device that transmits the stream so that the delay is within a second delay time that is smaller than a first delay time when the stream stored in the storage device is distributed, wherein when a transmission request requesting transmission of a stream to a receiving device is received from the second server device, the acquired first stream is relayed to both the first server device and the second server device, and when the transmission request is not received, the acquired first stream is relayed to the first server device; Processing section An information processing device comprising: (Configuration example 2) The processing unit a relay unit that relays the first stream; a switching unit that activates at least one of the relay unit and the second server device when the transmission request is received; The information processing device according to configuration example 1. (Configuration example 3) when the transmission request is not received, the processing unit relays the acquired first stream to the first server device but does not relay it to the second server device; The information processing device according to configuration example 1 or 2. (Configuration Example 4) The processing unit When the transmission request is received, determining whether the acquired stream can be transmitted so as to be delayed within the second delay time; adjusting a parameter of encoding by an encoding unit that encodes the stream when it is determined that the acquired stream cannot be transmitted so that the delay is within the second delay time. The information processing device according to any one of configuration examples 1 to 3. (Configuration Example 5) the encoding unit performs encoding using the adjusted parameters. The information processing device according to configuration example 4. (Configuration Example 6) the receiving device is a device that receives and displays the stream transmitted from the second server device, when the transmission request is received from the receiving device, the processing unit relays the acquired first stream to both the first server device and the second server device; 6. The information processing device according to any one of configuration examples 1 to 5. (Configuration Example 7) the processing unit relays the acquired first stream to both the first server device and the second server device when the transmission request is received from the transmission device. The information processing device according to any one of configuration examples 1 to 6. (Configuration Example 8) an information processing device according to any one of configuration examples 1 to 7; a third server device that acquires the first stream from the storage device of the first server device and distributes the acquired first stream, Information processing system. (Configuration Example 9) an information processing device according to any one of configuration examples 1 to 7; the first server device; The second server device, Information processing system. (Configuration Example 10) an information processing device according to any one of configuration examples 1 to 7; The transmission device, Information processing system. (Configuration Example 11) The transmission device a determination unit that, when the transmission request is received, determines whether the acquired stream can be transmitted so that the delay is within the second delay time; and an adjustment unit that adjusts a parameter of encoding by an encoding unit that encodes the stream when it is determined that the acquired stream cannot be transmitted so that the delay is within the second delay time. The information processing system according to configuration example 10. (Configuration Example 12) the encoding unit performs encoding using the adjusted parameters; The transmission device a transmission unit that transmits a stream encoded using the adjusted parameters to the information processing device; The information processing system according to configuration example 11. (Configuration Example 13) An information processing method executed by an information processing device, an acquisition step of acquiring a first stream transmitted from a transmission device; a first server device that stores an input stream in a storage device, and a second server device that transmits the stream so that the delay is within a second delay time that is smaller than a first delay time when the stream stored in the storage device is distributed; a relay step of relaying the acquired first stream to both the first server device and the second server device when a transmission request requesting a receiving device to transmit a stream is received from the second server device, and relaying the acquired first stream to the first server device when the transmission request is not received; An information processing method including: (Configuration Example 14) On the computer, an acquisition step of acquiring a first stream transmitted from a transmission device; a first server device that stores an input stream in a storage device, and a second server device that transmits the stream so that the delay is within a second delay time that is smaller than a first delay time when the stream stored in the storage device is distributed; a relay step of relaying the acquired first stream to both the first server device and the second server device when a transmission request requesting a receiving device to transmit a stream is received from the second server device, and relaying the acquired first stream to the first server device when the transmission request is not received; A program to execute.

[0115] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0116] 100, 100-2, 100-2b Information processing device 110, 110-2 Processing section 111 Acquisition Department 112 Relay Section 113, 113-2 Switching section 200 Cloud 210 Video Server 211 Storage device 220 Low Latency Server 230 Web Server 300, 400 receiving device 500, 500-2, 500-2b cameras 501 Encoding section 502 Transmission Unit 503-2 Judgment Department 504-2 Adjustment section

Claims

1. obtaining a first stream transmitted from a transmission device; a first server device that stores an input stream in a storage device, and a second server device that transmits the stream so that the delay is within a second delay time that is smaller than a first delay time when the stream stored in the storage device is distributed, wherein when a transmission request requesting transmission of a stream to a receiving device is received from the second server device, the acquired first stream is relayed to both the first server device and the second server device, and when the transmission request is not received, the acquired first stream is relayed to the first server device; Processing section An information processing device comprising:

2. The processing unit a relay unit that relays the first stream; a switching unit that activates at least one of the relay unit and the second server device when the transmission request is received. The information processing device according to claim 1 .

3. when the transmission request is not received, the processing unit relays the acquired first stream to the first server device but does not relay it to the second server device; The information processing device according to claim 1 .

4. The processing unit When the transmission request is received, determining whether the acquired stream can be transmitted with a delay within the second delay time; adjusting a parameter of encoding by an encoding unit that encodes the stream when it is determined that the acquired stream cannot be transmitted so that the delay is within the second delay time; The information processing device according to claim 1 .

5. the encoding unit performs encoding using the adjusted parameters. The information processing device according to claim 4 .

6. the receiving device is a device that receives and displays the stream transmitted from the second server device, the processing unit relays the acquired first stream to both the first server device and the second server device when the transmission request is received from the receiving device. The information processing device according to claim 1 .

7. the processing unit relays the acquired first stream to both the first server device and the second server device when the transmission request is received from the transmission device. The information processing device according to claim 1 .

8. The information processing device according to claim 1 ; a third server device that acquires the first stream from the storage device of the first server device and distributes the acquired first stream, Information processing system.

9. The information processing device according to claim 1 ; the first server device; The second server device, Information processing system.

10. The information processing device according to claim 1 ; The transmission device, Information processing system.

11. The transmission device a determination unit that, when the transmission request is received, determines whether the acquired stream can be transmitted so that the delay is within the second delay time; an adjusting unit that adjusts a parameter of encoding by an encoding unit that encodes the stream when it is determined that the acquired stream cannot be transmitted so that the delay is within the second delay time. The information processing system according to claim 10.

12. the encoding unit performs encoding using the adjusted parameters; The transmission device a transmission unit that transmits a stream encoded using the adjusted parameters to the information processing device; The information processing system according to claim 11.

13. An information processing method executed by an information processing device, an acquisition step of acquiring a first stream transmitted from a transmission device; a relay step of relaying the acquired first stream to both the first server device and the second server device when a transmission request for requesting a receiving device to transmit a stream is received from the second server device, the first server device storing an input stream in a storage device, and a second server device transmitting the stream so that the delay is within a second delay time that is smaller than a first delay time when the stream stored in the storage device is distributed, and relaying the acquired first stream to the first server device when the transmission request is not received; An information processing method including:

14. On the computer, an acquisition step of acquiring a first stream transmitted from a transmission device; a first server device that stores an input stream in a storage device, and a second server device that transmits the stream so that the delay is within a second delay time that is smaller than a first delay time when the stream stored in the storage device is distributed; a relay step of relaying the acquired first stream to both the first server device and the second server device when a transmission request requesting a receiving device to transmit a stream is received from the second server device, and relaying the acquired first stream to the first server device when the transmission request is not received; A program to execute.

Images