Video transmission system, video display control device and program

Abstract

To transmit and output moving images with a shorter delay than when transmitting and outputting moving images on a frame-by-frame basis.SOLUTION: A video transmission system 1 includes a transmitting device 10 that performs processing for acquiring video and transmitting the video as IP packets for each processing unit obtained by dividing a frame 201 that constitutes the video into units of a predetermined number of pixel lines, and a receiving device 30 that receives the IP packets and performs processing for each processing unit for converting the video into a signal that can be output on a display 70.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a moving image transmission system, a moving image display control device, and a program. [Background technology]

[0002] There is known a technique for transmitting moving images captured at a remote location and outputting them on a display (see, for example, Patent Document 1). In such a technique, the captured moving images are processed on a frame-by-frame basis. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-156653 A Summary of the Invention [Problem to be solved by the invention]

[0004] When the transmitted video is processed frame by frame, a delay of several frames occurs due to buffering, loss of synchronization between the video receiving device and the display, and the like.

[0005] An object of the present invention is to transmit and output moving images with less delay than when the moving images are transmitted and output on a frame-by-frame basis. [Means for solving the problem]

[0006] The present invention, which was completed with this objective in mind, is a moving image transmission system characterized by having a transmitting means which acquires moving images and transmits them as IP (Internet Protocol) packets for each processing unit obtained by dividing the frames that make up the moving images into units of a predetermined number of pixel lines, and a receiving and converting means which receives the IP packets and converts the moving images into signals that can be output on a display for each processing unit. Here, the receiving and converting means may generate a vertical synchronization signal for the display upon receiving the IP packet. Furthermore, each of the transmitting means and the receiving / converting means may be implemented in an FPGA (Field Programmable Gate Array). The communication device may further comprise a transmitting device having an FPGA in which the transmitting means is implemented, and a receiving device having an FPGA in which the receiving and converting means is implemented. Furthermore, the process by the transmitting means and the process by the receiving and converting means may be a series of processes, and the series of processes may be continuously performed by shifting the start timing for each processing unit. Moreover, the series of processes may be started for the processing unit, and when a predetermined time has elapsed, the series of processes may be started for another processing unit. Furthermore, if the series of processes for the processing unit is unsuccessful, the processing unit may be rewritten on the display based on past information. The receiving and converting means may rewrite the processing units on the display in the order in which the IP packets were transmitted. In addition, the transmitting means may perform processes of acquiring the frames of the moving image, correcting, encoding, and transmitting as the IP packets for each processing unit, and the receiving and converting means may perform processes of receiving the IP packets, decoding, and converting the frames into signals that can be output on the display for each processing unit. Furthermore, the header of the IP packet may include information indicating the position of the processing unit in the frame. The number of pixel lines aligned in the vertical direction of the frame may be a multiple of the number of pixel lines that constitute the processing unit. The present invention also relates to a moving image display control device, characterized in that it comprises a receiving means for receiving an IP packet in which a processing unit in which a frame constituting a moving image is divided into units of a predetermined number of pixel lines is transmitted as the IP packet, and a rewrite control means for generating a vertical synchronization signal for a display and rewriting the processing unit on the display when the IP packet of the processing unit is received by the receiving means. Here, the receiving means and the rewrite control means may be implemented in an FPGA. The present invention is also a program for enabling a computer to realize a function of acquiring moving images and transmitting them as IP (Internet Protocol) packets for each processing unit obtained by dividing the frames that make up the moving images into units of a predetermined number of pixel lines, and a function of receiving the IP packets and converting the moving images into signals that can be output on a display for each processing unit. Effect of the Invention

[0007] According to the present invention, moving images can be transmitted and output with less delay than when the moving images are transmitted and output on a frame-by-frame basis. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of the overall configuration of a moving image transmission system to which an embodiment of the present invention is applied; [Diagram 2] FIG. 2 illustrates an example of a hardware configuration of a transmitting device. [Diagram 3] 2 illustrates an example of a functional configuration of a control unit of a transmitting side device. FIG. [Figure 4] 10 illustrates an example of a functional configuration of a control unit of a receiving-side device. FIG. [Diagram 5] 13 is a flowchart showing an example of a process flow of a transmitting device. [Figure 6] 13 is a flowchart showing an example of a process flow of a receiving side device. [Figure 7]FIG. 1 is a simplified conceptual diagram showing the flow of a series of processes performed for each processing unit. [Figure 8] FIG. 13 is a diagram showing a specific example in which the number of pixel lines constituting a processing unit is 8 (lines). [Figure 9] FIG. 1 is a diagram showing a specific example of a conventional technique in which various processes are performed in units of frames. [Figure 10] 10 is a diagram showing a specific example of a frame-by-frame delay that occurs when the process of FIG. 9 is performed. [Figure 11] FIG. 1 is a diagram showing an overview of a technique for rewriting a display, as a conventional technique. [Figure 12] FIG. 1 is a diagram showing a specific example of a case where a display is rewritten by a conventional technique. [Figure 13] 11 is a diagram showing a specific example of a case where a display is rewritten under the control of a receiving device according to the present embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. <Configuration of video transmission system> FIG. 1 is a diagram showing an example of the overall configuration of a moving image transmission system 1 to which the present embodiment is applied. The video transmission system 1 is configured by connecting a transmitting device 10 having an FPGA (Field Programmable Gate Array) in which a transmitting means is implemented, and a receiving device 30 having an FPGA in which a receiving conversion means is implemented, via a network 90. ​​The network 90 is, for example, the Internet or a LAN (Local Area Network). A camera 50 is connected to the transmitting device 10 by wire or wirelessly. Also, a display 70 is connected to the receiving device 30 by wire or wirelessly. Each of the camera 50 and the display 70 may or may not be included in the video transmission system 1.

[0010] FPGAs are highly flexible hardware that allow users to rewrite circuits, and have the following advantages over, for example, CPUs (Central Processing Units). That is, FPGAs allow users to program the processes they need. This makes them flexible and allows for efficient parallel processing. In contrast, CPUs enable parallel processing through multi-cores (having multiple processor cores), but because they use the entire core, they require parallel processing even for unnecessary parts.

[0011] In addition, FPGAs enable direct access to data sources and respond at a constant speed using pre-installed logic. FPGAs also perform a series of processes at high speeds at a low clock frequency (e.g., 200 MHz) without using memory, using a special structure (e.g., pipelining). This makes it possible for FPGAs to achieve low latency of, for example, about 1 μs (microseconds). Furthermore, FPGAs are highly energy efficient because they use less circuit area. In contrast, CPUs perform a series of processes at a high clock frequency (e.g., 5 GHz) repeatedly using memory, which causes delays when the CPU is busy.

[0012] The transmitting device 10 and the receiving device 30 constituting the video transmission system 1 perform the following processing for each preset processing unit. One frame of a still image constituting a video (hereinafter referred to as a "frame") is composed of multiple rows of pixel lines arranged vertically. A "pixel line" is a horizontal line with 1 pixel in the vertical direction and n pixels in the horizontal direction (n is the number of horizontal pixels in one frame). A processing unit is obtained by dividing the multiple rows of pixel lines constituting a frame into units of a predetermined number of pixel lines. Therefore, the number of pixel lines constituting a processing unit is smaller than the number of pixel lines constituting a frame. The number of pixel lines constituting a processing unit is set so that the number of pixel lines constituting one frame is a multiple of the number of pixel lines constituting a processing unit. Specifically, for example, when the number of pixel lines in one frame is 720 (lines), the number of pixel lines constituting a processing unit is set to 8 (lines), 16 (lines), or the like.

[0013] That is, the transmitting device 10 performs a process of acquiring frames constituting a moving image captured by the camera 50 for each processing unit (hereinafter referred to as an "acquisition process"), a process of correcting the frames (hereinafter referred to as a "correction process"), a process of encoding the frames (hereinafter referred to as an "encoding process"), and a process of transmitting the frames as IP (Internet Protocol) packets (hereinafter referred to as a "packet transmission process").

[0014] Among these processes, the correction process adjusts, for example, saturation, brightness, hue, etc., for each processing unit. Also, the encoding process compresses and encrypts, for example, each processing unit according to a predetermined rule. The details and flow of these processes by the transmitting device 10 will be described later.

[0015] Furthermore, the receiving device 30 constituting the video transmission system 1 performs the following processes as a video display control device to which this embodiment is applied. That is, the receiving device 30 performs a process of receiving a processing unit transmitted as an IP packet from the transmitting device 10 (hereinafter referred to as a "packet receiving process"), a process of decoding (hereinafter referred to as a "decoding process"), and a process of converting into a signal that can be output by the display 70 (hereinafter referred to as a "conversion process"). Of these processes, the decoding process, for example, decodes the processing unit that has been coded by the encoding process described above.

[0016] The receiving device 30 also controls rewriting of the display 70. Specifically, the receiving device 30 controls rewriting of the display 70 by transmitting a vertical synchronization signal (VSYNC) to the display 70. Details of the control of rewriting of the display 70 by the receiving device 30 will be described later.

[0017] The receiving device 30 controls the rewriting of the display 70 for each processing unit. Specifically, the receiving device 30 generates a vertical synchronization signal (VSYNC) every time it acquires an IP packet for a processing unit transmitted from the transmitting device 10, and transmits it to the display 70. The receiving device 30 controls the rewriting of the display 70 (hereinafter referred to as "scanning process").

[0018] The receiving device 30 performs scanning in the order in which the IP packets were transmitted from the transmitting device 10. Specifically, the receiving device 30 performs scanning while reproducing as closely as possible the order in which the IP packets were transmitted by using a small amount of buffer. The header of an IP packet contains information indicating the position of the processing unit in the frame, frame identification information (number, etc.), information indicating the transmission order, and the like. Therefore, by referring to the header of the IP packet, it is possible to uniquely identify the position of the processing unit in the frame and the transmission order from the transmitting device 10.

[0019] When performing scanning, the receiving device 30 performs scanning based on, for example, past information for processing units in which the above-mentioned processes (packet reception process, decoding process, and conversion process) were not successful due to defects or the like. In this case, for example, the information used in the most recent scanning process can be reused as is. When the information used in the most recent scanning process is reused, the most recent state is maintained until the next scanning process is performed. In addition to a method of reusing past information as is, for example, a method of complementing by referring to information on surrounding pixels or a color close to the information on a pixel used in the past may be used. Furthermore, a method of complementing with only black or only white may be used.

[0020] Here, cases in which the above-mentioned processing is "unsuccessful" include, for example, when the IP packet does not reach the receiving device 30 for some reason, when the IP packet is delayed for some reason, when the processing by the transmitting device 10 is unsuccessful for some reason, when the processing by the receiving device 30 is unsuccessful for some reason, etc.

[0021] Among these, cases where the IP packet does not reach the receiving device 30 for some reason include cases where the IP packet is lost in a switch, router, or transmission path, cases where data is corrupted during transmission (bit inversion, etc.) and becomes an invalid packet, cases where the TTL (Time To Live) expires, etc. Also, cases where the IP packet is delayed for some reason include cases where a router or switch is congested, cases where the transmission path is congested, etc. It is also possible to adjust the numerical value for which delay is allowed.

[0022] In addition, the case where the processing by the transmitting device 10 is not successful for some reason includes, for example, when the memory load is high and memory access is delayed or fails, when the CPU load is high and processing is delayed or fails, when the SoC (System on chip) becomes hot and the speed is reduced, etc. In addition, the case where the processing by the receiving device 30 is not successful for some reason includes, for example, when the memory load is high and memory access is delayed or fails, when the CPU load is high and processing is delayed or fails, when the SoC becomes hot and the speed is reduced, etc. Details and flow of these processes by the receiving device 30 will be described later.

[0023] In the video transmission system 1, the above-mentioned processes (acquisition process, correction process, encoding process, and packet transmission process) by the transmitting device 10 and the above-mentioned processes (packet reception process, decoding process, and conversion process) by the receiving device 30 are considered to be a series of processes. The video transmission system 1 continuously performs a series of processes for each processing unit by shifting the start timing. That is, the video transmission system 1 starts a series of processes for a certain processing unit, and after a predetermined short time has passed, starts a series of processes for the next processing unit, repeating this like a "bucket brigade" so to speak, thereby transmitting video with low delay.

[0024] <Hardware configuration> (Hardware configuration of the sending device) FIG. 2 is a diagram illustrating an example of a hardware configuration of the transmitting side device 10. As shown in FIG. The transmitting device 10 has a control unit 11, a memory 12, a storage unit 13, a communication unit 14, an operation unit 15, and a display unit 16. These units are connected to each other via a data bus, an address bus, a PCI (Peripheral Component Interconnect) bus, etc.

[0025] The control unit 11 is a processor that controls the functions of the transmitting device 10 through the execution of various software such as an OS (operating system) and application software. The control unit 11 is configured with an FPGA or the like. The memory 12 is a storage area that stores various software and data used for executing the software, and is used as a working area for calculations. The memory 12 is configured with a RAM (random access memory), for example.

[0026] The storage unit 13 is a storage area for storing input data for various software programs and output data from various software programs. The storage unit 13 is composed of, for example, a hard disk drive (HDD), a solid state drive (SSD), a semiconductor memory, etc., used for storing programs and various setting data. A database for storing various information is stored in the storage unit 13. An example of the database stored in the storage unit 13 is a video DB 131 that can store video images.

[0027] The communication unit 14 transmits and receives data between the transmitting device 10 and the outside via the network 90. ​​The operation unit 15 is composed of, for example, a keyboard, a mouse, mechanical buttons, and switches, and accepts input operations. The operation unit 15 also includes a touch sensor that constitutes a touch panel integrally with the display unit 16. The display unit 16 is composed of, for example, a liquid crystal display or an organic EL (Electro Luminescence) display used to display information, and displays images, text data, and the like. A user interface and the like are displayed on the display unit 16.

[0028] (Hardware configuration of the receiving device) The hardware configuration of the receiving device 30 is similar to that of the transmitting device 10 shown in Fig. 2. That is, the receiving device 30 is equipped with a control unit, a memory, a storage unit, a communication unit, an operation unit, and a display unit that have the same functions as the control unit 11, the memory 12, the storage unit 13, the communication unit 14, the operation unit 15, and the display unit 16 shown in Fig. 2. For this reason, illustration and description of the hardware configuration of the receiving device 30 will be omitted.

[0029] <Functional configuration> (Functional configuration of the control unit of the transmitting device) FIG. 3 is a diagram showing an example of a functional configuration of the control unit 11 of the transmitting side device 10. As shown in FIG. In the control unit 11 of the transmitting side device 10, an acquisition unit 101, a correction unit 102, an encoding unit 103, and a transmission control unit 104 function.

[0030] The acquisition unit 101 acquires various kinds of information via the communication unit 14 (see FIG. 2). For example, the acquisition unit 101 performs the above-mentioned acquisition process. That is, the acquisition unit 101 acquires frames constituting a moving image captured by the camera 50 for each processing unit. The correction unit 102 performs the above-mentioned correction process. That is, the correction unit 102 adjusts the saturation, brightness, hue, and the like for each processing unit acquired by the acquisition process of the acquisition unit 101.

[0031] The encoding unit 103 performs the above-mentioned encoding process. That is, the encoding unit 103 performs encoding such as compression and encryption according to a predetermined rule for each processing unit corrected by the correction process of the correction unit 102. The transmission control unit 104, as a transmitting means, performs control for transmitting various information via the communication unit 14. For example, the transmission control unit 104 performs the above-mentioned packet transmission process. That is, the transmission control unit 104 performs control for transmitting each processing unit encoded by the encoding process of the encoding unit 103 as an IP packet.

[0032] (Functional configuration of the control unit of the receiving device) FIG. 4 is a diagram illustrating an example of a functional configuration of the control unit of the receiving side device 30. As shown in FIG. In the control section of the receiving side device 30, an acquisition section 301, a decoding section 302, a conversion section 303, a scanning control section 304, and a transmission control section 305 function. The acquisition unit 301, as a receiving unit of the reception conversion unit, acquires various information via the communication unit. For example, the acquisition unit 301 performs the above-mentioned packet reception process. That is, the acquisition unit 301 receives and acquires IP packets transmitted from the transmitting device 10 for each processing unit.

[0033] The decoding unit 302 performs the above-mentioned decoding process. That is, the decoding unit 302 decodes the processing unit that has been coded by the encoding process in the transmitting device 10. The conversion unit 303 performs the above-mentioned conversion process as a conversion unit of the reception conversion unit. That is, the conversion unit 303 converts the decoded processing unit into a signal that can be output on a display.

[0034] The scanning control unit 304 controls the rewriting (scanning) of the display 70. Specifically, the scanning control unit 304 controls the rewriting of the display 70 by generating a vertical synchronization signal (VSYNC) every time the acquisition unit 301 acquires an IP packet for each processing unit. The vertical synchronization signal (VSYNC) is a signal for rewriting the display 70 at regular intervals, and specifically, is a signal for instructing the start of rewriting the display 70.

[0035] The transmission control unit 305 performs control for transmitting various information via the communication unit. For example, the transmission control unit 305 performs control for transmitting, to the display 70, a signal that can be output to the display 70 and that is obtained as a result of the conversion process by the conversion unit 303. The transmission control unit 305 also performs control for transmitting, to the display 70, a vertical synchronization signal (VSYNC) generated by the scanning control unit 304.

[0036] <Processing flow of the sending device> FIG. 5 is a flowchart showing an example of the flow of processing performed by the transmitting device 10. As shown in FIG. When video images are transmitted from camera 50 (YES in step 501), transmitting device 10 acquires frames constituting the transmitted video images for each processing unit (step 502) as an acquisition process, and proceeds to the process of step 503. On the other hand, if video images are not transmitted from camera 50 (NO in step 501), transmitting device 10 repeats the determination process of step 501 until video images are transmitted from camera 50.

[0037] Next, the transmitting device 10 performs a correction process for each processing unit (step 503), and performs an encoding process for each processing unit (step 504). Next, the transmitting device 10 transmits an IP packet for each processing unit (step 505) as a packet transmission process.

[0038] When the processing for each processing unit for all frames constituting the moving image is completed (YES in step 506), the transmitting side device 10 completes the processing on the transmitting side (END). On the other hand, when the processing for each processing unit for all frames constituting the moving image is not completed (NO in step 506), the processing of the transmitting side device 10 returns to the judgment processing of step 501.

[0039] <Processing flow of the receiving device> FIG. 6 is a flowchart showing an example of the flow of processing performed by the receiving device 30. As shown in FIG. When the receiving device 30 receives an IP packet for each processing unit from the transmitting device 10 (YES in step 601), the receiving device 30 receives the IP packet for each processing unit (step 602) and generates a vertical synchronization signal (step 603) as an acquisition process. On the other hand, if the transmitting device 10 has not transmitted an IP packet for each processing unit (NO in step 601), the receiving device 30 repeats the determination process of step 601 until the transmitting device 10 transmits an IP packet for each processing unit.

[0040] Next, the receiving device 30 performs a decoding process by decoding each processing unit (step 604), and performs a conversion process by converting each processing unit (step 605). Specifically, the signal is converted into a signal that can be outputted to the display .

[0041] If the series of processes for each processing unit is successful (YES in step 606), the receiving device 30 performs rewriting by transmitting the vertical synchronization signal generated in step 603 to the display 70 as a scanning process (step 607), and proceeds to the determination process of step 609. On the other hand, if the series of processes for each processing unit is unsuccessful (NO in step 606), the receiving device 30 does not rewrite the display 70 for the processing unit for which the series of processes was unsuccessful (step 608), and proceeds to the determination process of step 609.

[0042] When the processing for each processing unit for all frames constituting the moving image is completed (YES in step 609), the receiving side device 30 completes the receiving side processing (END). On the other hand, when the processing for each processing unit for all frames constituting the moving image is not completed (NO in step 609), the processing of the receiving side device 30 returns to the judgment processing in step 601.

[0043] <Example> Fig. 7 is a simplified conceptual diagram of a series of processing steps performed for each processing unit. Fig. 7 shows an example in which one frame 201, out of a plurality of frames constituting a moving image, is divided into eight (block) processing units 211 to 218. Note that the example of division shown in Fig. 7 is simplified to make the explanation easier to understand, and in reality, for example, the frame may be divided into a large number of processing units (90 (blocks) in the example of Fig. 8) as in the example of Fig. 8 described later.

[0044] When a video captured by the camera 50 is transmitted to the transmitting device 10, the transmitting device 10 performs acquisition, correction, encoding, and packet transmission processes for each processing unit. When each processing unit is IP transmitted, the receiving device 30 performs packet reception, decoding, and conversion processes for each processing unit. In the example of Fig. 7, the above series of processes for each of the processing units 211 to 218 are performed consecutively in that order with different start times.

[0045] FIG. 8 is a diagram showing a specific example in which the number of pixel lines constituting a processing unit is 8 (lines). In FIG. 8, the vertical axis represents the number of processing units (number of blocks), and the horizontal axis represents the flow of time. The values ​​on the horizontal axis ("1 pixel", "8 lines") represent the length of delay due to buffering. For example, "8 lines" on the horizontal axis indicates that a delay of "8 lines" occurs due to buffering. In addition, the size of a frame constituting a moving image is assumed to be 1280 (pixels) in the horizontal direction and 720 (pixels) in the vertical direction. In addition, the number of pixel lines constituting a processing unit is assumed to be 8 (lines), and the number of processing units is assumed to be 90 (blocks) (720 (pixels)÷8 (lines)=90 (blocks)).

[0046] 8 shows a part of a series of processes for each of 3 (blocks) processing units 211 to 213, which are a part of 90 (blocks). As described above, the video transmission system 1 performs a series of processes for each of a plurality of processing units successively with staggered start timings. For this reason, in the example of FIG. 8, when the acquisition process for processing unit 211 ends and the correction process starts, the acquisition process for processing unit 212 starts. Thereafter, when the acquisition process for processing unit 212 ends and the correction process starts, the acquisition process for processing unit 213 starts. Such processes are performed successively for 90 (blocks).

[0047] FIG. 9 is a diagram showing a specific example of a conventional technique in which various types of processing are performed in units of frames. FIG. 10 is a diagram showing a specific example of a frame-by-frame delay that occurs when the process of FIG. 9 is performed. In Fig. 9, the vertical axis represents the number of frames constituting a video image, and the horizontal axis represents the flow of time. The value "1 Frame (720 Lines)" on the horizontal axis represents the length of delay due to buffering, and indicates that a delay of "1 Frame (720 Lines)" occurs due to buffering. In the example of Fig. 9, the size of a frame is 1280 pixels in the horizontal direction and 720 pixels in the vertical direction, similar to the example of Fig. 8 described above.

[0048] In the conventional technology, various types of processing are performed on a frame-by-frame basis that constitutes a moving image. For example, as shown in FIG. 9, when the number of pixels in the vertical direction of a frame is 720 (lines), various types of processing are performed for every 720 (lines). Specifically, a process for acquiring one frame (1Frame) is started, and when the process is completed, a process for correcting one frame (1Frame) is started. Then, when the process is completed, a process for encoding one frame (1Frame) is started. Then, when the process is completed, a process for transmitting one frame (1Frame) as an IP packet (IP transmission) is started. In the conventional technology, such processing is repeated for the number of frames that constitute a moving image.

[0049] As described above, in conventional technology, various processes are performed on a frame-by-frame basis, and therefore delays due to buffering occur on a frame-by-frame basis. "Buffering" refers to adjusting for differences in processing speed and processing time by temporarily storing frames in a buffer memory (DRAM (Dynamic Random Access Memory) in the example of Fig. 10) to prevent video images from being interrupted by processing delays. Hereinafter, frames temporarily stored in memory by buffering will be referred to as "frame buffer."

[0050] As shown in Fig. 10, a conventional transmitting device performs buffering between acquiring a moving image transmitted from a camera and correcting it. As a result, in the conventional transmitting device, a one-frame delay occurs between acquiring a moving image and correcting it. Also, a conventional transmitting device performs buffering between correcting a moving image and encoding it. As a result, in the conventional transmitting device, a one-frame delay occurs between correcting a moving image and encoding it. Thereafter, the conventional transmitting device encodes the moving image, temporarily stores the encoded data in a buffer memory (DRAM), and then transmits it as an IP packet (IP transmission).

[0051] As shown in Fig. 10, a conventional receiving device receives (IP reception) video transmitted as IP packets (IP transmission), temporarily stores the video in a buffer memory (DRAM) as a receiving buffer, and then decodes the video. The conventional receiving device then buffers the decoded video until it is converted into a signal that can be output on a display. For this reason, in the conventional receiving device, a one-frame delay occurs until the decoded video is converted.

[0052] Furthermore, the conventional receiving device performs control on a frame-by-frame basis to output the moving image converted into a signal that can be output by the display to the display. For this reason, in the conventional receiving device, a delay of up to one frame occurs between the time when a frame of the moving image is converted into a signal that can be output by the display and the time when it is output to the display. For these reasons, in the conventional technology, a delay of up to four frames may occur between the time when the moving image transmitted from the camera passes through the conventional receiving device and the time when it is output to the display.

[0053] <Display rewriting> FIG. 11 is a diagram showing an overview of a conventional technique for rewriting a display. The display is rewritten by rewriting the horizontal lines (hereafter referred to as "scan lines") arranged in multiple rows (hundreds to thousands) vertically on the display, in order from the top row to the bottom. When the bottom row has been rewritten, the rewriting starts again from the top row. This process is repeated to update the display.

[0054] A scanning line is a line created by extremely small light spots moving at high speed, and for example, the display is rewritten by moving the light spot in the direction of the arrow as shown in Figure 11. In other words, the display is rewritten by moving a light spot starting from the left end of the top row of the display in the direction of the arrow to the right end of the bottom row as the goal.

[0055] The movement of the light spot is controlled by inputting a horizontal synchronization signal (HSYNC) and a vertical synchronization signal (VSYNC) to the display. The horizontal synchronization signal (HSYNC) is a synchronization signal for moving the light spot to the left end of the row below. The vertical synchronization signal (VSYNC) is a synchronization signal for moving the light spot that has reached the right end of the bottom row (goal position) to the left end of the top row (start position). In other words, the vertical synchronization signal plays a role in determining both the timing to start rewriting from the start position and the number of scanning line rows to be rewritten.

[0056] FIG. 12 is a diagram showing a specific example of a case where a display is rewritten by the conventional technique. In the conventional technology, the process of converting video frames into signals that can be output by a display and the process of transmitting frames temporarily stored in a display buffer to a display are in an independent (asynchronous) relationship. For this reason, depending on the timing at which the vertical synchronization signal (VSYNC) is input to the display, a delay of up to one frame may occur.

[0057] For example, as shown in the upper diagram of Fig. 12, frames of moving images converted by a conventional receiving device into signals that can be output by a display are temporarily stored in a display buffer. Then, when a process of transmitting a vertical synchronization signal (VSYNC) generated by the conventional receiving device to the display is started, transmission of frames from the display buffer to the display is started.

[0058] Transmission of a frame from the display buffer to the display is performed by rewriting the display scan lines from top to bottom, as shown in the lower diagram of Figure 12. Here, a conventional receiving device performs a process of converting the next frame to be stored in the display buffer as image preparation while a frame is being transmitted from the display buffer to the display. After that, if transmission of the frame from the display buffer to the display is not yet complete when the conversion process is completed, the device goes into a waiting state until frame transmission is completed. This causes a delay (a delay of up to one frame).

[0059] FIG. 13 is a diagram showing a specific example of rewriting the display 70 under the control of the receiving-side device 30 according to the present embodiment. As described above, in the conventional technology, a maximum delay of one frame occurs between when a video frame is converted into a signal that can be output by a display and when it is output to the display. This is because the process of converting a video frame into a signal that can be output by a display and the process of transmitting the frame temporarily stored in the display buffer to the display are independent (asynchronous).

[0060] In contrast, the receiving device 30 according to the present embodiment synchronizes the process of converting the frames of a moving image into signals that can be output by the display 70 for each processing unit with the process of transmitting the frames temporarily stored in the display buffer to the display 70. Specifically, as shown in the upper diagram of Fig. 13, the receiving device 30 transmits a vertical synchronization signal (VSYNC) to the display 70 in accordance with the timing at which the processing units acquired as IP packets are converted into signals that can be output by the display 70 and temporarily stored in the display buffer. This causes the vertical synchronization signal (VSYNC) to be periodically reset.

[0061] When the vertical synchronization signal (VSYNC) is periodically reset, the timing of the following two processes is synchronized, as shown in the lower diagram of Fig. 13. That is, it is possible to always synchronize the timing at which the processing unit acquired by the receiving device 30 as an IP packet is converted into a signal that can be output by the display 70 and temporarily stored in the display buffer, and the timing at which the frame temporarily stored in the display buffer is transmitted to the display 70 (the timing at which rewriting of the scan lines of the display 70 starts). This makes it possible to suppress delays occurring between when a moving image frame is converted into a signal that can be output by the display 70 and when it is output to the display.

[0062] In summary, the moving image transmission system 1 according to the present invention only needs to have the following configuration, and can take various forms. In other words, the video transmission system 1 is a video transmission system characterized by having a transmitting device 10 that performs the process of acquiring video and transmitting it as IP packets for each processing unit obtained by dividing the frames that make up the video into units of a predetermined number of pixel lines, and a receiving device 30 that receives the IP packets and performs the process of converting the video into a signal that can be output on a display for each processing unit.

[0063] This allows a series of processes, including a process of acquiring a moving image and transmitting it as an IP packet, and a process of receiving the IP packet and converting the moving image into a signal that can be output on a display, to be performed for each processing unit obtained by dividing a frame that constitutes the moving image into units of a predetermined number of pixel lines. As a result, the series of processes for each processing unit are performed continuously with different start timings, so that the moving image can be transmitted and output with less delay than when the moving image is transmitted and output on a frame-by-frame basis.

[0064] Here, the receiving device 30 may generate a vertical synchronization signal (VSYNC) for the display 70 upon receiving the IP packet for processing. Thereby, in accordance with the timing at which the processing unit acquired as an IP packet and converted is stored in the display buffer, a vertical synchronization signal (VSYNC) is generated and transmitted to the display 70. As a result, the vertical synchronization signal (VSYNC) is periodically reset, so that the timing at which the converted processing unit is stored in the display buffer can be synchronized with the timing at which rewriting of the scan lines of the display 70 starts.

[0065] Furthermore, the acquisition unit 101, the acquisition unit 301, and the conversion unit 303 may be implemented in an FPGA. Also, the transmitting side device 10 may include an FPGA on which the acquisition unit 101 is implemented, and a receiving side device 30 may include an FPGA on which the acquisition unit 301 and the conversion unit 303 are implemented. This makes it possible to build a video transmission system that is more flexible and efficient, achieves low latency, and is highly energy efficient, compared to systems controlled by a CPU, for example.

[0066] Furthermore, the processing by the transmitting device 10 and the processing by the receiving device 30 may be treated as a series of processing, and the series of processing may be performed consecutively by shifting the timing at which the series of processing starts for each processing unit. Also, a series of processes may be started for a processing unit, and when a predetermined time has elapsed, a series of processes may be started for another processing unit. This allows a series of processes including the processes by the transmitting device 10 and the receiving device 30 to be performed continuously with the start timing shifted for each processing unit (for example, every time a predetermined time has elapsed). As a result, the moving image can be transmitted and output with less delay than when the moving image is transmitted and output in frame units.

[0067] Furthermore, if a series of processes for a processing unit is not successful, the processing unit may be rewritten on the display 70 based on past information. This allows rewriting (scanning) to be performed based on past information, so the past state (for example, the most recent state) is maintained until the next rewriting (scanning) is performed, preventing any loss of a part of the frame.

[0068] Furthermore, the receiving device 30 may rewrite the display 70 in units of processing in the order in which the IP packets were transmitted. This maintains the order in which the scan lines of the display 70 are rewritten.

[0069] In addition, the transmitting device 10 may perform processes of acquiring frames of video, correcting them, encoding them, and transmitting them as IP packets for each processing unit, and the receiving device 30 may perform processes of receiving IP packets, decoding them, and converting the frames into signals that can be output on a display for each processing unit. This allows a series of processes including the process by the transmitting device 10 and the process by the receiving device 30 to be performed continuously with the start timing shifted for each processing unit. As a result, the video can be transmitted and output with less delay than when the video is transmitted and output in frame units.

[0070] Furthermore, the header of the IP packet may contain information indicating the position of the processing unit in the frame. This makes it possible to uniquely identify the position of the processing unit in the frame and the order in which the packets were transmitted from the transmitting device 10 to the receiving device 30 by referencing the header of the IP packet.

[0071] Furthermore, the number of pixel lines aligned in the vertical direction of a frame may be a multiple of the number of pixel lines constituting a processing unit. This allows processing in which the frame is divided evenly.

[0072] Moreover, the receiving side device 30 as the moving image display control device according to the present invention only needs to have the following configuration, and can take various forms. In other words, the receiving device 30 is a moving image display control device characterized by having an acquisition unit 301 that receives IP packets in which processing units, in which frames constituting a moving image are divided into units of a predetermined number of pixel lines, are transmitted as IP packets, and a conversion unit 303 that generates a vertical synchronization signal (VSYNC) for the display 70 and rewrites the processing unit on the display 70 when the acquisition unit 301 receives the IP packet of the processing unit.

[0073] Thereby, in accordance with the timing at which the processing unit acquired as an IP packet and converted is stored in the display buffer, a vertical synchronization signal (VSYNC) is generated and transmitted to the display 70. As a result, the vertical synchronization signal (VSYNC) is periodically reset, so that the timing at which the converted processing unit is stored in the display buffer can be synchronized with the timing at which rewriting of the scan lines of the display 70 starts.

[0074] Here, the acquisition unit 301 and the conversion unit 303 may be implemented in an FPGA. This makes it possible to configure the receiving device 30 as a video display control device that is more flexible and efficient, can achieve low delay, and is highly energy efficient, as compared to a system controlled by a CPU, for example.

[0075] The present invention is also a program for enabling a computer to realize a function of acquiring moving images and transmitting them as IP (Internet Protocol) packets for each processing unit obtained by dividing the frames that make up the moving images into units of a predetermined number of pixel lines, and a function of receiving the IP packets and converting the moving images into signals that can be output on a display for each processing unit.

[0076] This allows a series of processes, including a process of acquiring a moving image and transmitting it as an IP packet, and a process of receiving the IP packet and converting the moving image into a signal that can be output on a display, to be performed for each processing unit obtained by dividing a frame that constitutes the moving image into units of a predetermined number of pixel lines. As a result, the series of processes for each processing unit are performed continuously with different start timings, so that the moving image can be transmitted and output with less delay than when the moving image is transmitted and output on a frame-by-frame basis.

[0077] <Other embodiments> Although the present embodiment has been described above, the present invention is not limited to the above-described present embodiment. Moreover, the effects of the present invention are not limited to those described in the above-described present embodiment. For example, the configuration of the video transmission system 1 shown in FIG. 1, the hardware configuration of the transmitting device 10 shown in FIG. 2, the functional configuration of the control unit 11 of the transmitting device 10 shown in FIG. 3, and the functional configuration of the control unit of the receiving device 30 shown in FIG. 4 are merely examples for achieving the object of the present invention, and are not particularly limited. In other words, it is sufficient that the video transmission system 1 shown in FIG. 1 has a function capable of executing the above-described processing as a whole, and the hardware configuration and functional configuration used to realize this function are not limited to the above-described examples.

[0078] In addition, the order of the steps of the process of the transmitting device 10 shown in Fig. 5 and the order of the steps of the process of the receiving device 30 shown in Fig. 6 are merely examples and are not particularly limited. The processes are not limited to the chronological order of the steps shown in the figures, and may be performed in parallel or individually. In addition, the specific example shown in Fig. 13 is also merely an example and is not particularly limited.

[0079] For example, in the above embodiment, the acquisition process, correction process, encoding process, and packet transmission process are given as processes performed by the transmitting device 10 for each processing unit, but these processes are merely examples. Processes other than these processes may be performed by the transmitting device 10 for each processing unit, or at least one of these processes may be performed by each processing unit. Also, the packet reception process, decoding process, and conversion process are given as processes performed by the receiving device 30 for each processing unit, but these processes are also merely examples. Processes other than these processes may be performed by the receiving device 30 for each processing unit, or at least one of these processes may be performed by each processing unit.

[0080] In the above embodiment, a plurality of pixel lines are treated as a processing unit, but a plurality of pixels constituting a pixel line may be treated as a processing unit. That is, a part or all of a pixel line may be treated as a processing unit. When a plurality of pixel lines are treated as a processing unit as in the above embodiment, a two-dimensional compression process of, for example, 8 (pixels) x 8 (pixels) is performed, whereas when a part of a pixel line is treated as a processing unit, a one-dimensional compression process of, for example, 8 (pixels) is performed. In this case, since the compression rate of the two-dimensional compression process is higher than that of the one-dimensional compression process, the amount of data increases when a part of a pixel line is treated as a processing unit. [Explanation of symbols]

[0081] 1...video image transmission system, 10...transmitting device, 11...control unit, 12...memory, 13...storage unit, 14...communication unit, 15...operation unit, 16...display unit, 30...receiving device, 50...camera, 70...display, 101...acquisition unit, 102...correction unit, 103...encoding unit, 104...transmission control unit, 201...frame, 211, 212, 213, 214, 215, 216, 217, 218...processing unit, 301...acquisition unit, 302...decoding unit, 303...conversion unit, 304...scanning control unit, 305...transmission control unit, 90...network

Claims

1. a transmission means for acquiring a moving image and transmitting the acquired moving image as an Internet Protocol (IP) packet for each processing unit obtained by dividing a frame constituting the moving image into a unit of a predetermined number of pixel lines; a receiving and converting means for receiving the IP packets and converting the moving image into a signal that can be output on a display for each processing unit; A moving image transmission system comprising:

2. The receiving and converting means generates a vertical synchronization signal for the display when the IP packet is received.

2. The video transmission system according to claim 1.

3. Each of the transmitting means and the receiving / converting means is implemented in an FPGA (Field Programmable Gate Array).

3. The moving image transmission system according to claim 1.

4. a transmitting side device including an FPGA in which the transmitting means is implemented, and a receiving side device including an FPGA in which the receiving conversion means is implemented, 4. The moving image transmission system according to claim 3.

5. The process by the transmitting means and the process by the receiving and converting means are performed as a series of processes, and the start timing of the series of processes is shifted for each processing unit to perform the processes continuously.

2. The video transmission system according to claim 1.

6. the series of processes for the processing unit is started, and when a predetermined time has elapsed, the series of processes for another processing unit is started, 6. The moving image transmission system according to claim 5.

7. When the series of processes for the processing unit is not successful, the processing unit is rewritten on the display based on past information.

6. The moving image transmission system according to claim 5.

8. The receiving and converting means rewrites the processing units on the display in the order in which the IP packets were transmitted.

6. The moving image transmission system according to claim 5.

9. the transmitting means performs a process of acquiring the frames of the moving image, a process of correcting the frames, a process of encoding the frames, and a process of transmitting the frames as IP packets for each processing unit; the receiving and converting means performs a process of receiving the IP packet, a process of decoding the IP packet, and a process of converting the frame into a signal that can be output by the display for each processing unit.

2. The video transmission system according to claim 1.

10. a header of the IP packet includes information indicating a position of the processing unit in the frame; 2. The video transmission system according to claim 1.

11. the number of pixel lines arranged in the vertical direction of the frame is a multiple of the number of pixel lines constituting the processing unit, 2. The video transmission system according to claim 1.

12. a receiving means for receiving an IP packet, the IP packet being a processing unit obtained by dividing a frame constituting a moving image into a unit of a predetermined number of pixel lines; a redraw control means for generating a vertical synchronization signal for a display when the IP packet of the processing unit is received by the receiving means, and controlling redrawing of the processing unit on the display; A moving image display control device comprising:

13. The receiving means and the rewrite control means are implemented in an FPGA. The moving image display control device according to claim 12.

14. On the computer, A function of acquiring a moving image and transmitting the acquired moving image as an Internet Protocol (IP) packet for each processing unit obtained by dividing a frame constituting the moving image into a unit of a predetermined number of pixel lines; a function of receiving the IP packets and converting the moving image into a signal that can be output on a display for each of the processing units; A program to achieve this.

Images