Web server device, video distribution system, video distribution method, and program

Abstract

The present invention provides a web server device and the like that can reduce latency when distributing video without requiring redundant communication lines. [Solution] The web server device comprises a frame buffer, a frame processing unit, and a transmission unit. The frame buffer temporarily stores at least a time series of pairs of frame images and time codes representing the time information of the frame images. The frame processing unit reads the frame images within a predetermined time period from the frame buffer by referring to the time codes, and encodes the read frame images using a predetermined encoding scheme. The transmission unit transmits the frame images encoded by the frame processing unit to the client device.

Description

Technical Field

[0001] The present invention relates to a web server device, a video distribution system, a video distribution method, and a program.

Background Art

[0002] Technologies for distributing video using a communication network such as the Internet have been put into practical use. When distributing video using a communication network, the amount of delay in transmission or the like can be a problem.

[0003] As a conventional technology, protocols such as FLV (Adobe Flash Video) and HLS (HTTP Live Streaming) are widely used for streaming distribution of video. For example, the standard delay when using HLS is about 30 seconds to 45 seconds. It is desirable to reduce such a delay.

[0004] Also, in the invention described in Patent Document 1, it is recognized that a delay may occur in displaying emergency information transmitted during a disaster or the like. Further, in the invention described in Patent Document 1, in order to solve that problem, the distribution system uses a separate transmission path apart from the transmission path for video and audio content. That is, the distribution system transmits emergency information in an emergency such as a disaster through a transmission path different from the transmission path for transmitting normal video and audio content.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The present invention aims to reduce the transmission delay of video and other data. However, when using the technology described in Patent Document 1, it is necessary to configure redundant transmission lines to reduce the delay, which leads to the problem of increased communication network costs.

[0007] This invention was made based on the above-mentioned problem recognition, and aims to provide a web server device, a video distribution system, a video distribution method, and a program that can reduce delays when distributing video without providing redundant communication lines. This is the purpose. [Means for solving the problem]

[0008] [1] In order to solve the above problems, a web server device according to one aspect of the present invention comprises a frame buffer that stores at least temporarily a time series of pairs of frame images and time codes representing time information of the frame images; a frame processing unit that reads the frame images within a predetermined time period from the frame buffer by referring to the time codes and encodes the read frame images using a predetermined encoding scheme; and a transmission unit that transmits the frame images encoded by the frame processing unit to a client device.

[0009] [2] In addition, in the web server device described in [1] above, the frame processing unit reads only the latest frame image from the frame buffer and encodes the read frame image using a predetermined encoding method.

[0010] [3] Another aspect of the present invention is a web server device according to [1] or [2] above, wherein the transmission unit establishes and maintains a connection with the client device and sequentially transmits a plurality of frame images through the established connection.

[0011] [4] Another aspect of the present invention is that, in the web server device described in [3] above, the transmission unit transmits the frame image to the client device using a WebSocket.

[0012] [5] Another aspect of the present invention is that in any of the web server devices described in [1] to [4] above, the frame processing unit encodes the frame image using the JPEG (Joint Photographic Experts Group) encoding method.

[0013] [6] Another aspect of the present invention is a web server device according to any of the above [1] to [5], wherein the frame buffer stores a plurality of time series of pairs of frame images and time codes, the frame processing unit reads the frame images from the frame buffer for each series within a predetermined time, encodes the read frame images using a predetermined encoding method, and the transmission unit transmits the encoded frame images for each series in parallel.

[0014] [7] Another aspect of the present invention is a video distribution system comprising any of the web server devices described in [1] to [6] above, and a client device that receives the encoded frame images transmitted from the web server device and sequentially renders the received plurality of frame images as a bitmap image.

[0015] [8] Another aspect of the present invention is a video distribution method in which a frame buffer stores at least temporarily a time series of pairs of frame images and time codes representing time information of the frame images, a frame processing unit reads frame images within a predetermined time period from the frame buffer by referring to the time codes, encodes the read frame images using a predetermined encoding scheme, and a transmission unit transmits the frame images encoded by the frame processing unit to a client device.

[0016] [9] Another aspect of the present invention is a program for causing a computer to function as a web server device, comprising: a frame buffer that at least temporarily stores a time series of pairs of frame images and time codes representing time information of the frame images; a frame processing unit that reads the frame images within a predetermined time period from the frame buffer by referring to the time codes and encodes the read frame images using a predetermined encoding scheme; and a transmission unit that transmits the frame images encoded by the frame processing unit to a client device. [Effects of the Invention]

[0017] According to the present invention, low-latency video distribution can be achieved without the need to provide redundant communication lines. [Brief explanation of the drawing]

[0018] [Figure 1] This block diagram shows the schematic functional configuration of a video distribution system according to an embodiment of the present invention. [Figure 2] This block diagram shows the schematic internal functional configuration of the video signal processing device according to the same embodiment. [Figure 3] This is a block diagram showing the approximate internal functional configuration of the web server device according to the same embodiment. [Figure 4] This is a schematic diagram showing an example of a sequence of pairs of frame images and timecodes that are at least temporarily stored in the frame buffer in the embodiment. [Figure 5] This is a schematic diagram showing an example of a state in which a sequence of pairs of frame images and time codes is stored in the frame buffer in the same embodiment. [Figure 6] This is a schematic diagram showing another example of a state in which a sequence of pairs of frame images and time codes is stored in the frame buffer in the same embodiment. [Figure 7]It is a schematic diagram showing an example of a communication sequence between a web server device and a web browser in the same embodiment. [Figure 8] It is a flowchart showing the procedure of the operation of the web server device according to the same embodiment. [Figure 9] It is a block diagram showing an example of the internal configuration of a video signal processing device, a web server device, and a client device in the same embodiment. [Figure 10] It is a schematic diagram showing the features of the first modification example of the same embodiment. [Figure 11] It is a schematic diagram showing the features of the second modification example of the same embodiment.

Mode for Carrying Out the Invention

[0019] Next, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below enables video distribution with low latency. The reasons for requiring low latency are as follows, for example.

[0020] (1) Watching TV broadcasts and app viewing For example, in the relay distribution of a sports event or the like, the viewer may refer to information different from the TV broadcast on a second screen app (the app is an abbreviation for "application program") together with the TV broadcast. At this time, a delay also occurs in the TV broadcast. Therefore, it is important for the second screen app to match the timing of video display to the delay of the TV broadcast. In order for the second screen app to match the timing of the TV broadcast, it is necessary to deliver the information (video, etc.) of the second screen app with low latency. By delivering the information of the second screen app with low latency, it is possible to avoid problems such as missing the timing of exchanging opinions about the result of a sports game, for example.

[0021] (2) Video chat For example, when a reporter interviews someone in a remote location within a video, bidirectional delays can cause long silences or lead to both parties speaking at the same time. Empirically, it is known that to ensure a smooth conversation, the upper limit of bidirectional delay should be kept below approximately 150 milliseconds (≒ 1 / 7 second).

[0022] (3) Races and auctions Activities such as sports betting and online auctions require speed, and streaming as close to real-time as possible is essential. For example, in online auctions, even a slight delay can cause bids to not be recorded properly or result in different winning bids.

[0023] (4) Online video games Timing is crucial in online video games. A video transmission delay of 100 milliseconds or less is strongly desired. When playing games via streaming services, achieving low latency eliminates various problems (for example, attacking enemies that no longer exist in the game).

[0024] In the embodiments described below, video signal frame data is converted into binary data and streamed to the browser on the client device. The embodiments use the RTSP and WebSocket protocols. RTSP stands for "Real-Time Streaming Protocol." RTSP is a protocol standardized by the IETF (Internet Engineering Task Force) for controlling the distribution (streaming) of real-time data. WebSocket is a communication standard for bidirectional communication between a web server and a browser. WebSocket enables bidirectional communication while maintaining an online state between the server and the client (user).

[0025] Figure 1 is a block diagram illustrating the schematic functional configuration of a video distribution system according to this embodiment. As shown in the figure, the video distribution system 1 comprises a video signal processing device 2, a web server device 3, and a client device 4.

[0026] At least some of the functions of the video signal processing device 2, the web server device 3, and the client device 4 can be implemented, for example, by a computer and a program. Each functional unit also has storage means as needed. These storage means may include, for example, program variables or memory allocated during program execution. Alternatively, non-volatile storage means such as magnetic hard disk drives or solid-state drives (SSDs) may be used as needed. Furthermore, at least some of the functions of each device may be implemented as dedicated electronic circuits rather than by a computer and a program.

[0027] The video distribution system 1 is a system for distributing video. Specifically, the web server device 3 transmits video, which is received by the client device 4 and displayed.

[0028] The video signal processing device 2 passes a sequence of frame data to the web server device 3 based on the video signal obtained from an external source.

[0029] The web server device 3 distributes at least a portion of the frame data received from the video signal processing device 2 as video to the client device 4.

[0030] Client device 4 receives video (a series of frame data) from web server device 3 and displays it on a screen or the like. Client device 4 includes a web browser 41 internally. The web browser 41 is implemented using a computer program and an environment for running that program. Client device 4 may be, for example, a PC (personal computer), a smartphone, or a wearable device.

[0031] The web browser 41 sequentially draws the sequence of frame data received from the web server device 3 onto the screen as a bitmap image.

[0032] In other words, client device 4 receives encoded frame images transmitted from web server device 3 and sequentially renders the received multiple frame images as a bitmap image.

[0033] With the video distribution system 1 having the above configuration, the web browser 41 of the client device 4 receives the video distribution and displays the video on a screen or the like. The video signal processing device 2 and the web server device 3 are connected by a communication line or the like. Furthermore, the web server device 3 and the client device 4 can communicate bidirectionally via a communication network such as the Internet.

[0034] Figure 2 is a block diagram showing the schematic internal functional configuration of the video signal processing device. As shown in the figure, the video signal processing device 2 is composed of a video processing unit 21 and an RTSP client function unit 22.

[0035] The video processing unit 21 processes the video signal received from an external source and outputs a sequence of frame images.

[0036] The RTSP client function unit 22 communicates with the web server device 3 as an RTSP client. The RTSP client function unit 22 sequentially transmits a sequence of frame images output from the video processing unit 21 to the web server device 3.

[0037] Figure 3 is a block diagram showing the schematic functional configuration inside the web server device. As shown in the figure, the web server device 3 is composed of an RTSP server function unit 31, a frame buffer 32, a frame processing unit 33, and a web socket server function unit 34.

[0038] The RTSP server function unit 31 sequentially receives a sequence of frame images from the RTSP client function unit 22 of the video signal processing device 2. The RTSP server function unit 31 sequentially writes the received frame images to the frame buffer 32.

[0039] The frame buffer 32 temporarily stores a time series of pairs of frame images and time codes representing the time information of the frame images. The frame buffer 32 is implemented using storage means such as semiconductor memory.

[0040] The frame processing unit 33 reads and processes frame images from the frame buffer 32. Specifically, by referring to the time code, the frame processing unit 33 reads the frame images from the frame buffer that are from a predetermined time period retrospectively from the present moment, or the most recent frame image at the present moment, and encodes the read frame images using a predetermined encoding scheme.

[0041] When the frame processing unit 33 encodes a frame image, it may, for example, use the JPEG (Joint Photographic Experts Group) encoding scheme. Images encoded with JPEG are so-called binary data.

[0042] The WebSocket server function unit 34 transmits the frame image encoded by the frame processing unit 33 to the client device 4. The WebSocket server function unit 34 can transmit the frame image to the client device 4 using the WebSocket method described above. The WebSocket server function unit 34 may also be referred to as the "transmission unit".

[0043] More specifically, the WebSocket server function unit 34 (transmitting unit) establishes and maintains a connection with the client device 4. The WebSocket server function unit 34 sequentially transmits a plurality of the frame images to the client device 4 via the established connection.

[0044] The timecode for each frame image may be added to each frame image by the RTSP server function unit 31. Alternatively, the timecode for each frame image may be added to each frame image by the video processing unit 21 or the RTSP client function unit 22 within the video signal processing device 2. Alternatively, the timecode for each frame image may already be added to each frame image in the video signal input to the video signal processing device 2.

[0045] Figure 4 is a schematic diagram showing an example of a sequence of frame image and timecode pairs that are at least temporarily stored in the frame buffer 32. As shown in the figure, the frame buffer 32 can hold multiple frame images. Each frame image is accompanied by a timecode. The timecode is information that at least represents the order of the frame images. The timecode can be expressed in a format such as "YYYYMMDD HHMMSSnnn" (year, month, day, hour, minute, second, sequential number (nnn)). In the example shown in the figure, each frame image and timecode pair is conveniently assigned a number (1, 2, 3, 4). In the figure, pairs from the 5th onward are omitted.

[0046] Figure 4 shows only the frame image and timecode pairs up to number 4, but the RTSP server function unit 31 sequentially writes the received frames to the frame buffer 32. In other words, the number of frame image and timecode pairs continues to increase even after the state shown in Figure 4. Alternatively, the web server device 3 may delete frame image and timecode pairs in the frame buffer 32 as needed once it has finished processing.

[0047] Figure 5 is a schematic diagram showing an example of a state in which a sequence of frame image and time code pairs is stored in the frame buffer 32. In the illustrated example, frame image and time code pairs 123, 124, 125, and 126 are stored in the frame buffer 32. Pairs 122 and earlier are omitted in the figure. In the illustrated state, pair 126 contains the most recent frame image. Frame images after frame image 126 have not yet been stored in the frame buffer 32. In other words, frame images after frame image 126 have not yet arrived.

[0048] In the state shown in Figure 5, the frame processing unit 33 reads only the frame image no. 126 as the target for processing. In other words, the frame processing unit 33 reads only the most recent frame image at that time as the target for processing. That is, when frame image no. 126 is stored in the frame buffer as the latest image, the previous frame images (125, 124, 123, etc.) are not targeted for processing, regardless of whether they have already been processed by the frame processing unit 33 or not.

[0049] Figure 6 is a schematic diagram showing another example of a state in which a series of pairs of frame images and time codes are stored in the frame buffer 32. In the illustrated example, pairs of frame images and time codes numbered 131, 132, 133, and 134 are stored in the frame buffer 32. In this figure, pairs prior to number 130 are omitted. In the illustrated state, the time codes of pairs 132, 133, and 134 indicate times within a predetermined time period in the past from the present time. The difference between the time indicated by the time code of pair 131 and the present time is greater than the predetermined time period. In other words, pairs prior to number 131 are older pairs. Also, frame images after frame image 134 have not yet been stored in the frame buffer 32.

[0050] In the state shown in Figure 6, the frame processing unit 33 reads only the unprocessed frame images from among the frame images 132, 133, and 134 mentioned above. In other words, the frame processing unit 33 reads only the unprocessed frame images from the frame images within a predetermined time period, working backward from the present moment. That is, the frame processing unit 33 reads the frame images within the predetermined time period from the frame buffer 32 by referring to the time code of the pair, and encodes the read frame images using a predetermined encoding method. In other words, each of the frame images prior to the predetermined time period (frame 131 and earlier) is not subject to processing at the present moment, regardless of whether or not they have already been processed by the frame processing unit 33.

[0051] As described above, the frame processing unit 33 processes only unprocessed frame images from the current time to a predetermined time period. In other words, the frame processing unit 33 may skip processing older unprocessed frame images. Alternatively, the frame processing unit 33 may process only the most recent frame image at that time. In other words, the frame processing unit 33 may skip processing older frame images.

[0052] By performing the processing as described above, the web server device 3 does not consume resources to process delayed frame images. In other words, the web server device 3 achieves real-time performance (low latency) in video distribution.

[0053] Figure 7 is a schematic diagram showing an example of a communication sequence between the web server device 3 and the web browser 41.

[0054] As shown in the diagram, in step S101, the web server device 3 and the web browser 41 first establish a communication connection between them. This connection is also called a "session." Once this connection is established, the web server device 3 and the web browser 41 can send data to each other.

[0055] After the connection is established, in step S102, the web server device 3 sends the first frame image to the web browser 41. The web browser 41 can then display the received frame image on its screen.

[0056] Similarly, in subsequent steps S103, S104, etc., the web server device 3 sequentially transmits frame images to the web browser 41. Upon receiving these frame images, the web browser 41 can display the received frame images on its screen.

[0057] Once the transmission of a series of frame images is complete (for example, the end of the video content to be delivered), the web server device 3 and the web browser 41 may disconnect their connection (step S105).

[0058] Note that the communication procedure described with reference to Figure 7 is just one example. The procedure does not necessarily have to be the same as in Figure 7, as long as the web server device 3 can sequentially transmit frame images to the client device 4. For example, the client device 4 may send ACK (acknowledgment) information regarding the reception of frame images to the web server device 3. Alternatively, the client device 4 may send some data other than ACK information upstream to the web server device 3. Alternatively, some data other than frame images may be sent downstream (from the web server device 3 to the client device 4).

[0059] Figure 8 is a flowchart showing the operation procedure of the web server device 3. The operation procedure of the web server device 3 will be explained below in accordance with this flowchart.

[0060] First, in step S1, the web server device 3 determines whether or not to terminate the process. The process is terminated, for example, when the video distribution ends or when an instruction to terminate the process is entered by the operator. If the process is terminated (step S1: YES), the entire process of this flowchart is terminated. If the process is not terminated (step S1: NO), the process proceeds to the next step S2.

[0061] Next, in step S2, the web server device 3 waits for the arrival of the frame image. For example, the frame processing unit 33 can monitor the frame buffer 32 to determine whether the frame image has arrived. Alternatively, the RTSP server function unit 31 may notify the frame processing unit 33 of the arrival of the frame image.

[0062] Next, in step S3, the frame processing unit 33 of the web server device 3 refers to the time code and retrieves only the most recent frame image from the frame images stored in the frame buffer 32. Alternatively, the frame processing unit 33 refers to the time code and retrieves only the frame images stored in the frame buffer 32 that are from a predetermined time period retrospectively from the current time.

[0063] Next, in step S4, the frame processing unit 33 of the web server device 3 encodes the frame image extracted in step S3 in JPEG format. The frame processing unit 33 then passes the resulting JPEG-formatted frame image to the web socket server function unit 34.

[0064] Next, in step S5, the web socket server function unit 34 of the web server device 3 transmits the frame image encoded in step S4 to the web browser 41 running on the client device 4 using WebSocket. After the processing in step S5 is completed, the process returns to step S1.

[0065] Figure 9 is a block diagram showing examples of the internal configurations of the video signal processing device 2, web server device 3, and client device 4 in the embodiment. Each device can be implemented using a computer. As shown in the figure, the computer consists of a central processing unit 901, RAM 902, input / output ports 903, input / output devices 904 and 905, etc., and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions contained in programs read from RAM 902, etc. The central processing unit 901 writes data to RAM 902, reads data from RAM 902, and performs arithmetic and logical operations according to each instruction. RAM 902 stores data and programs. Each element contained in RAM 902 has an address and can be accessed using that address. RAM stands for "Random Access Memory". Input / output ports 903 are ports for the central processing unit 901 to exchange data with external input / output devices, etc. Input / output devices 904 and 905 exchange data with the central processing unit 901 via input / output port 903. Bus 906 is a common communication channel used within the computer. For example, the central processing unit 901 reads and writes data to RAM 902 via bus 906. Also, for example, the central processing unit 901 accesses input / output port 903 via bus 906.

[0066] Furthermore, at least some of the functions of the video signal processing device 2, web server device 3, and client device 4 in the above-described embodiment can be implemented using a computer and a program. In this case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, DVD-ROMs, USB memory, and storage devices such as hard disks built into a computer system. In other words, "computer-readable recording medium" may be a non-transitory computer-readable recording medium. Moreover, "computer-readable recording medium" may also include those that temporarily and dynamically hold programs, such as communication lines when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in such a case. Furthermore, the above-mentioned program may be for implementing some of the functions described above, and may also be able to implement the above-mentioned functions in combination with a program already recorded in the computer system.

[0067] Although embodiments have been described above, the present invention can also be implemented in the following modified forms. Furthermore, multiple modified forms may be combined to the extent possible.

[0068] [First variation] Figure 10 is a schematic diagram illustrating the first modified example. As shown in the figure, in the first modified example, the web server device 3 transmits video to multiple client devices 4 (three in the illustrated example). In this first modified example, the operation of the RTSP server function unit 31, the frame buffer 32, and the frame processing unit 33 is as already described with reference to Figures 9 and earlier. In the first modified example, the web socket server function unit 34 of the web server device 3 establishes a connection with each of the multiple client devices 4 and transmits a sequence of frame images to each client device 4 using RTSP. In other words, the web server device 3 distributes the same video to each of the multiple client devices 4.

[0069] [Second variation] Figure 11 is a schematic diagram illustrating a second modified example. As shown in the diagram, in the second modified example, the web server device 3 processes multiple video signals in parallel. The illustrated example shows the first, second, and third video signals. In reality, the number of video signal sequences that the web server device 3 processes simultaneously is arbitrary.

[0070] In the illustrated example, the RTSP server function unit 31 receives the first, second, and third video signals from an external source. The RTSP server function unit 31 writes the frame images contained in each video signal to the frame buffer 32. The frame buffer 32 manages the frame images of the first, second, and third series as frame images of another series and stores them at least temporarily. In other words, when the frame processing unit 33 refers to the frame buffer 32, it can determine, based on the time code in each series, which frame image of the first, second, and third series is from a predetermined time period retrospectively from the present moment, or which is the latest frame image at the present moment. For each series, the frame processing unit 33 retrieves the frame image from the frame buffer 32 from a predetermined time period retrospectively from the present moment, or which is the latest frame image at the present moment, and encodes it in JPEG format. For each series, the frame processing unit 33 sequentially passes the encoded frame images to the websocket server function unit 34. The WebSocket server function unit 34 sequentially transmits each series of frame images to the web browser 41 of the client device 4.

[0071] In other words, in the second modification, the frame buffer 32 stores, at least temporarily, time series of pairs of frame images and time codes for multiple series. The frame processing unit 33, in parallel, reads from the frame buffer 32, for each of those series, the frame images from a predetermined time period going backward from the present, or the latest frame image at the present time, and encodes the read frame images using a predetermined encoding scheme. The frame processing unit 33 passes the encoded frame images to the web socket server function unit 34 (transmitter). The web socket server function unit 34 (transmitter) transmits the encoded frame images to the client device 4, series by series, in parallel. The client device 4, having received a series of frame images, displays those frame images sequentially on the screen. In other words, the client device 4 displays the video transmitted from the web server device 3 on the screen.

[0072] The web server device 3 may transmit multiple different video sequences (sequences of frame images) to the same client device 4, or it may transmit them to multiple separate client devices 4.

[0073] [Third variation] As a third modification, the order of the processes in step S3 and step S4 of the processing procedure shown in Figure 8 may be reversed. In other words, the processing procedure in the case of the third modification is as follows.

[0074] In step S2, the web server device 3 waits for the arrival of frame images. When frame images arrive, the frame processing unit 33 of the web server device 3 encodes all of the frame images in JPEG format. The web server device 3 stores the JPEG encoded frame images in the frame buffer 32, associating them with the timecode of the frame image. Next, the frame processing unit 33 of the web server device 3 retrieves only the most recent frame image from the frame images stored in the frame buffer 32 by referring to the timecode. At this time, the frame processing unit 33 retrieves the JPEG encoded frame image from the frame buffer 32. The frame processing unit 33 passes the retrieved JPEG frame image to the web socket server function unit 34. The processing from step S5 onward is the same as the processing described with reference to Figure 8.

[0075] In the case of the third modified example, the frame processing unit 33 encodes the latest frame image read from the frame buffer 32 using a predetermined encoding method.

[0076] In the third embodiment, instead of the frame processing unit 33 retrieving only the latest frame image from the frame images stored in the frame buffer 32 and passing it to the web socket server function unit 34, the frame processing unit 33 may retrieve multiple frame images having time codes within a predetermined time difference from the current time and pass them to the web socket server function unit 34.

[0077] As described above, according to the embodiment (including modifications), the frame processing unit 33 of the web server device 3 processes only the frame image within a predetermined time or the most recent frame image at that time, skipping other frame images. Furthermore, the frame processing unit 33 encodes the frame images in JPEG. As a result, no dependencies between frame images occur in the encoding of the data to be transmitted. These features enable the web server device 3 to deliver video with very little delay.

[0078] The embodiments described above can be understood as follows.

[0079] [1] A web server device according to one embodiment includes a frame buffer that stores at least temporarily a time series of pairs of frame images and time codes representing time information of the frame images; a frame processing unit that reads the frame images within a predetermined time period from the frame buffer by referring to the time codes and encodes the read frame images using a predetermined encoding scheme; and a transmission unit that transmits the frame images encoded by the frame processing unit to a client device. With this configuration, the frame processing unit can skip older frame images that have already arrived and process only frame images within a predetermined time period.

[0080] [2] In another aspect of the present invention, in the web server device described in [1] above, the frame processing unit reads only the latest frame image from the frame buffer and encodes the read frame image using a predetermined encoding scheme. With this configuration, the frame processing unit can process only the latest frame image.

[0081] [3] Another aspect of the present invention is a web server device according to [1] or [2] above, wherein the transmitting unit establishes and maintains a connection with the client device and sequentially transmits a plurality of frame images through the established connection. With this configuration, it is not necessary to re-establish the connection each time a frame image is transmitted.

[0082] [4] Another aspect of the present invention is that, in the web server device described in [3] above, the transmission unit transmits the frame image to the client device using a WebSocket.

[0083] [5] Another aspect of the present invention is that in any of the web server devices described in [1] to [4] above, the frame processing unit encodes the frame image using the JPEG (Joint Photographic Experts Group) encoding scheme. With this configuration, there is no dependency between frame images in terms of encoding scheme. In other words, even if one frame image is missing for any reason, it does not affect the distribution and display of subsequent frame images.

[0084] [6] Another aspect of the present invention is a web server device according to any of the above [1] to [5], wherein the frame buffer stores time series of pairs of frame images and time codes for a plurality of series, the frame processing unit reads the frame images from the frame buffer for a predetermined time for each series in parallel, encodes the read frame images using a predetermined encoding method, and the transmission unit transmits the encoded frame images for each series in parallel. With this configuration, the web server device can distribute a plurality of video series (sequences of frame images).

[0085] [7] Another aspect of the present invention is a video distribution system comprising any of the web server devices described in [1] to [6] above, and a client device that receives the encoded frame images transmitted from the web server device and sequentially renders the received plurality of frame images as a bitmap image.

[0086] [8] Another aspect of the present invention is a video distribution method in which a frame buffer stores at least temporarily a time series of pairs of frame images and time codes representing time information of the frame images, a frame processing unit reads frame images within a predetermined time period from the frame buffer by referring to the time codes, encodes the read frame images using a predetermined encoding scheme, and a transmission unit transmits the frame images encoded by the frame processing unit to a client device.

[0087] [9] Another aspect of the present invention is a program for causing a computer to function as a web server device comprising: a frame buffer that at least temporarily stores a time series of pairs of frame images and time codes representing time information of the frame images; a frame processing unit that reads the frame images within a predetermined time period from the frame buffer by referring to the time codes and encodes the read frame images using a predetermined encoding scheme; and a transmission unit that transmits the frame images encoded by the frame processing unit to a client device. By using this program, a general-purpose computer can be made to function as any of the web server devices described in [1] to [6] above.

[0088] [Performance measurement results] The inventors developed the present invention with the goal of achieving video distribution with a delay of less than 0.5 seconds. When a prototype of the video distribution system 1 according to the present embodiment was actually built and operated, the measurement result showed a delay of 0.2 seconds. In other words, the effectiveness of this embodiment was proven.

[0089] Although embodiments of this invention (including modifications) have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Industrial applicability]

[0090] The present invention can be used, for example, for video distribution. However, the scope of use of the present invention is not limited to those exemplified herein. [Explanation of Symbols]

[0091] 1. Video distribution system 2. Video signal processing device 3. Web server device 4. Client devices 21 Video Processing Unit 22 RTSP Client Function Unit 31 RTSP Server Function Unit 32 frame buffers 33 Frame Processing Unit 34. WebSocket Server Function Unit (Transmitting Unit) 41 Web browser 901 Central Processing Unit 902 RAM 903 Input / Output Ports 904,905 Input / Output Devices 906 Bus

Claims

1. A frame buffer that at least temporarily stores a time series of pairs of frame images and time codes representing the time information of the frame images, A frame processing unit that, by referring to the aforementioned time code, does not process frame images prior to a predetermined time from the present moment, reads frame images from the frame buffer within a predetermined time period from the present moment, and encodes the read frame images using a predetermined encoding method, A transmission unit that transmits the frame image encoded by the frame processing unit to a client device, A web server device equipped with the following features.

2. The frame processing unit reads only the latest frame image from the frame buffer and encodes the read frame image using a predetermined encoding method. The web server device according to claim 1.

3. The transmitting unit establishes and maintains a connection with the client device, and sequentially transmits a plurality of frame images through the established connection. The web server device according to claim 1.

4. The transmitting unit transmits the frame image to the client device using a WebSocket. The web server device according to claim 3.

5. The frame processing unit encodes the frame image using the JPEG (Joint Photographic Experts Group) encoding method. The web server device according to claim 1.

6. The frame buffer stores time series of pairs of frame images and time codes for multiple series. The frame processing unit, in parallel, reads the frame images from the frame buffer for each sequence within a predetermined time, encodes the read frame images using a predetermined encoding method, The transmitting unit transmits the encoded frame image for each sequence in parallel. The web server device according to claim 1.

7. The web server device described in claim 1, A client device that receives the encoded frame images transmitted from the web server device and sequentially renders the received multiple frame images as a bitmap image, A video distribution system equipped with the following features.

8. The frame buffer temporarily stores at least a time series of pairs of frame images and time codes representing the time information of the frame images. The frame processing unit, by referring to the time code, does not process frame images prior to a predetermined time from the current time, reads frame images from the frame buffer within a predetermined time period from the current time, encodes the read frame images using a predetermined encoding method, The transmitting unit transmits the frame image encoded by the frame processing unit to the client device. Video distribution methods.

9. A frame buffer that at least temporarily stores a time series of pairs of frame images and time codes representing the time information of the frame images, A frame processing unit that, by referring to the aforementioned time code, does not process frame images prior to a predetermined time from the present moment, reads frame images from the frame buffer within a predetermined time period from the present moment, and encodes the read frame images using a predetermined encoding method, A transmission unit that transmits the frame image encoded by the frame processing unit to a client device, A web server device equipped with a program to make a computer function as such.

Images