VIDEO DELIVERY SYSTEM, VIDEO DELIVERY METHOD, TRANSMISSION DEVICE, AND TRANSMISSION PROGRAM
The system addresses inefficient video distribution by packetizing data with identifiers for frames and scanning lines, enabling selective reception based on network and display capabilities, ensuring high-quality distribution across multiple devices.
Patent Information
- Application Number
- JP2022532220
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-06-26
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-06-26
AI Technical Summary
Existing methods for distributing uncompressed video data over networks struggle with insufficient network bandwidth and display device compatibility, leading to inefficient distribution to multiple display devices and potential network congestion.
The system packetizes video data with different identifiers for frames and scanning lines, allowing selective reception based on network bandwidth and display capabilities, using existing network packet headers to manage traffic.
Enables efficient transfer of large video data to remote display devices with varying capabilities via multiple networks, ensuring high-quality video distribution without network overload and device compatibility issues.
Smart Images

Figure 0007586178000001 
Figure 0007586178000002 
Figure 0007586178000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a system for distributing a screen to a remote location via a network. [Background technology]
[0002] Conventionally, connection methods that assume short-distance connections of several meters to several tens of meters have been used to transfer video data to display devices such as displays and monitors. For example, HDMI (High-Definition Multimedia Interface) (registered trademark) [Non-Patent Document 1] is a method that has become widespread for connecting computers and their monitors, or televisions and related AV (Audio / Visual) devices, but it is not possible to directly connect devices in remote locations, such as several kilometers away, and transfer video data.
[0003] To transmit video data to a remote location, a device is used to convert the HDMI signal into a signal suitable for communication. One method known is to convert the HDMI signal into IP (Internet Protocol) packets in a specific format [Non-Patent Documents 2, 3]. [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] https: / / www.hdmi.org / spec / hdmi2_1 [Non-Patent Document 2] https: / / www.aja-jp.com / news / press / 1068-aja-announces-ipt-10g2-hdmi-and-ipt-10g2-sdi [Non-Patent Document 3] http: / / h-path.co.jp / service_product / 3-monitoring-with-transmitter-receiver / Summary of the Invention [Problem to be solved by the invention]
[0005] When transmitting video data to a remote location, one video source may need to be simultaneously distributed to multiple display devices, not just one. In particular, to achieve low-latency distribution, it is necessary to transmit uncompressed video data.
[0006] Conventional methods for transmitting uncompressed video data over a network, such as HDMI, have not been able to efficiently distribute one video source to multiple display devices. For example, the same video data is only distributed to all display devices by multicast. Such systems have two problems.
[0007] One is that there may not be enough network bandwidth to transmit uncompressed video data to all receiving devices. If distribution is performed at a uniform high image quality and high frame rate, the amount of uncompressed video data will be huge, and not only will video content not be displayed on networks with insufficient bandwidth, but it will also put strain on the network bandwidth and impede other communications. In addition, distribution based on a low network bandwidth will cause the problem of not being able to take advantage of the quality of the high image quality and high frame rate video source.
[0008] Another problem is that there may not be a display device that can display video compatible with the pixel and frame rate of the video source. When the video source is not a fixed system but is dynamically changed each time, or when distributing to multiple display devices, it is difficult for all display devices to be compatible with all video source formats.
[0009] In other words, an object of the present disclosure is to enable the transfer of large volumes of video data to display devices in remote locations with different display capabilities via multiple networks with different available bandwidths. [Means for solving the problem]
[0010] A transmitting side transmits video data in packets with different identifiers for each frame or scan line, and a receiving side selectively receives packets with a predetermined identifier according to the network bandwidth or the image quality that can be displayed.
[0011] Specifically, the video distribution system of the present disclosure includes: a transmitting device packetizing the video data for each frame or each scan line, and transmitting the packets by writing a plurality of identifiers in a header; A receiving device selectively receives video data having a header in which a predetermined identifier from among the plurality of identifiers is written.
[0012] Specifically, the video distribution method of the present disclosure includes: a transmitting device packetizing the video data for each frame or each scan line, and transmitting the packets by writing a plurality of identifiers in a header; A receiving device selectively receives video data having a header in which a predetermined identifier from among the plurality of identifiers is written.
[0013] Specifically, the transmitting device of the present disclosure includes: a signal analyzer for identifying at least one of a frame and a line of video data; a packetizing unit that packetizes video data for at least one of a frame and a scan line; a data sending unit that writes a plurality of identifiers in a header and sends each packet generated by the packetizing unit; Equipped with.
[0014] The transmission program of the present disclosure is a program for causing a computer to realize each functional unit of the transmission device of the present disclosure, and is a program for causing a computer to execute each step of the transmission method performed by the transmission device of the present disclosure. Effect of the Invention
[0015] According to the present disclosure, large volumes of video data can be transferred to display devices in remote locations with different display capabilities via multiple networks with different available bandwidths. [Brief description of the drawings]
[0016] [Figure 1] 1 shows an example of a schematic configuration of the present disclosure. [Diagram 2] An example of address control using frames is shown below. [Diagram 3] 1 shows an example of address control using scanning lines. [Figure 4] An example of address control using something other than image data will be described. [Diagram 5] 1 shows a first arrangement example of audio data. [Figure 6] 13 shows a second example of audio data arrangement. [Figure 7] 1 illustrates an example system configuration of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings are assumed to indicate the same components.
[0018] (Specific explanation of the developed technology) With reference to FIG. 1, the schematic configuration of the video distribution system of the present disclosure will be described. The transmitting side transmits one video source with different identifiers used for distribution depending on the type of data such as pixels and number of frames and the order of frames, and the receiving side selects the identifier and receives the packet, thereby obtaining video data of the required number of pixels and frames. This allows the transmitting side to selectively receive and display the video data according to the network bandwidth of the receiving side and the image quality that can be displayed by the display device by transmitting only one video source. In particular, even if the receiving side has multiple or many receiving devices or display devices, or multiple networks are interposed between the transmitting side and the receiving side, the present invention does not require multiple video sources, making it a system that is easily expandable.
[0019] The identifier used for the above-mentioned distribution is any identifier written in the header of a packet that can be read by general network devices such as routers and switches, and examples of such identifiers include the following: (1) Ethernet (registered trademark): destination MAC (Media Access Control) address, source MAC address (2) IP (Internet Protocol): Destination address, Source address, ToS (3) TCP / UDP (Transmission Control Protocol / User Datagram Protocol): Source port number, destination port number (4) VLAN (Virtual Local Area Network) (802.1Q, 802.1ad): VLAN ID, Priority (PCP) (5) VXLAN (Virtual eXtensible Local Area Network): VXLAN ID (VNI), outer identifier equivalent to (1-4) (6) NVGRE (Network Virtualization using Generic Routing Encapsulation): Tenant Network Identifier (TNI), outer identifier equivalent to (1-4) (7)STT (Spin Torque Transfer): Context ID (8) MPLS (Multi-Protocol Label Switching): Label, EXP (Experimental) bit (9) PPPoE (Point-to-Point Protocol over Ethernet): Session ID (10) L2TP (Layer 2 Tunneling Protocol): Tunnel ID, Session ID
[0020] When an address is used as an identifier, the address may be either or both of a source address and a destination address. A multicast address may also be used for the destination address. In addition to an address, a different identifier contained in the header of a packet or the like may also be used. For example, in the case of a TCP / IP packet, a port number may be used. In this way, any identifier can be adopted according to the protocol used for the packet.
[0021] We will now discuss the case where an IP address is used as an identifier. For example, for one screen of data, the multicast address 239.0.0.1 is used as the destination address for packets carrying pixel data for even-numbered driving lines, and the multicast address 239.0.0.2 is used as the destination address for packets carrying pixel data for odd-numbered driving lines, and the receiving side can selectively receive only 239.0.0.1, only 239.0.0.2, or both 239.0.0.1 and 239.0.0.2. In addition, the multicast address can be used separately from the receiving side by the network side to transfer packets only to routes with receiving terminals using MLD (Multicast Listener Discovery) or IGMP (Internet Group Management Protocol), making it possible to prevent unnecessary packet transmission to routes with no receiving terminals.
[0022] In other words, the method makes it possible to realize selective distribution of content using general network devices such as routers and switches that can read packet header information, without using a device that deeply understands the contents of data, such as a DPI (Deep Packet Inspection) device. In the following, this disclosure will explain the case where the identifier is an address.
[0023] Consider a case where a transmitting device 101 distributes data to a receiving device 102 via a network 121, a network boundary device 112, and a network 122, and at the same time distributes data to a receiving device 103 via a network 121, a network boundary device 113, and a network 123. A single video source is distributed from 101 using addresses A and B, which differ depending on the pixel and frame.
[0024] If 122 and 102 are capable of receiving high image quality and high frame rate, the content can be played back using one original video source by receiving both addresses A and B. On the other hand, if 123 or 103 can only transfer and receive low image quality and low frame rate, 103 will receive data from only one address.
[0025] If the address is multicast, only data of the address selected by 103 is transferred from 113 to 123, so that 123 and 103 are not overloaded.
[0026] When the display capability of the display device 132 is high image quality and high frame rate, the receiving device 102 can play back the content using the single original video source by receiving both addresses A and B. On the other hand, when the display capability of the display device 133 is low image quality and low frame rate, the receiving device 103 receives data from only one of addresses A and B.
[0027] With reference to Figure 2, we will explain address control using frames. The sending side sends the 2kth frame with address A, and the 2k+1th frame with address B. In other words, the addresses {A, B} are repeated for each frame.
[0028] A receiver capable of receiving a high frame rate receives Address A and Address B. A receiver capable of receiving only at a low frame rate selectively receives only Address A or Address B. For example, if the transmitted video data is 60 fps, receiving Address A and B will allow reception at 60 fps, but receiving only Address A or B will halve the amount of data and allow reception at 30 fps.
[0029] Although the above describes a case where the data amount and frame rate are reduced to 1 / 2, the present invention is not limited to this. For example, by setting it to {A,B,A,C}, if all of A, B, and C are received, it can be received at the original frame rate, if only A or B and C are received, it can be received at half the data amount and frame rate, and if only B or C is received, it can be received at 1 / 4 the data amount and frame rate. For example, by setting it to {A,B,C}, if all of A, B, and C are received, it can be received at the original frame rate; if A and B or B and C or A and C are received, it can be received at 2 / 3 of the data volume and frame rate; and if only A or B or C is received, it can be received at 1 / 3 of the data volume and frame rate.
[0030] The video data used in this disclosure is premised on video data that contains all information for one image at a certain time, and is intended for uncompressed video or video with a relatively low compression rate, such as Motion JPEG (Joint Photographic Experts Group) that does not use a compression method between frames. In addition, the use of identifiers within one screen is premised on progressive uncompressed video.
[0031] With reference to Figure 3, we will explain how to control addresses using scan lines. The transmitting side sends address A for the 2kth scan line and address B for the 2k+1th scan line. In other words, the addresses {A, B} are sent repeatedly for each scan line.
[0032] A receiver capable of receiving high image quality receives address A and address B. A receiver capable of receiving only low image quality selectively receives either address A or address B. For example, if the number of scanning lines in the transmitted video is 2160, receiving address A and B will allow it to be received as 2160, but receiving only address A or B will halve the amount of data and allow it to be received as 1080.
[0033] Although the above describes a case where the amount of data and the number of lines are reduced to 1 / 2, the present invention is not limited to this. For example, by setting it to {A,B,A,C}, if all of A, B, and C are received, the original number of lines will be received; if only A or B and C are received, half the amount of data and number of lines will be received; and if only B or C is received, 1 / 4 the amount of data and number of lines will be received. For example, by setting {A,B,C}, if all of A, B, and C are received, the original number of lines will be received; if A and B or B and C or A and C are received, 2 / 3 of the data amount and number of lines will be received; and if only A or B or C is received, 1 / 3 of the data amount and number of lines will be received.
[0034] With reference to Fig. 4, we will explain the case where addresses are controlled using data other than image (pixel) data. 400 shows the entire data series for one screen when transferring video data. The area 402, which is part of the entire data series, is the pixel data itself and is subject to the above control, while the remaining area 401 is other data such as audio.
[0035] Pattern 1 The information such as audio can be the same data regardless of the image quality and frame rate of the received data. For this reason, an address different from the address used in 402 can be used for the data in the area of 401. By receiving the data from a common address only for 401 regardless of the image quality and frame rate of the received data, it becomes possible to receive audio without interruption.
[0036] Pattern 2 The information such as audio can be the same data regardless of the image quality and frame rate received. To achieve this, the area of 401 is controlled in the same way as the address of 402, and audio information can be moved to a data area that is commonly received. For example, if video is divided into {A, B} and there is a receiving device that receives (A, B) and a receiving device that receives only (A), the audio data, etc. originally included in B can be moved to the data area of the A address. This makes it possible to receive audio, etc. without interruption.
[0037] Fig. 5 shows an example of the arrangement of audio data in part of the area 401. If the area 401 is controlled in the same way as the address of 402, for example, if there is a terminal that receives only even-numbered line data, there will be loss of audio data. Therefore, by concentrating only the audio data on the even lines when transmitting, as shown in Fig. 6, it is possible to receive audio data without loss even when only the even lines are received. The same is true in the case of selective reception of screen frames. Also, on the receiving side, it is possible to return the audio data to its original position, or not.
[0038] An example of a system configuration of the present disclosure will be described with reference to Fig. 7. Reference numeral 500 denotes a transmitting device of this system, and reference numeral 600 denotes a receiving device. Reference numeral 500 transmits video data input from a camera or computer via an interface such as HDMI, HD-SDI, USB, or GigE to a network, and reference numeral 600 receives the video data from 500 and outputs it to a television or monitor via an interface such as HDMI, HD-SDI, USB, or GigE.
[0039] (Transmitting device) Reference numeral 501 denotes an input buffer, from which video data is read out from 502 and 503 at the required timing. A signal analysis unit 502 reads the video data from 501, identifies frames, scanning lines, audio data, and other control signals, and notifies 503 and 504. The data itself may be passed to 503. 503 is a data adjustment unit, which reads video data from 501 or 502, replaces and rewrites some of the data, such as audio data, based on the information from 502, and passes the data to 504. At this time, marker data may be embedded so that it is clear that the data has been rewritten. A packetizing unit 504 reads the video data from 503, and packetizes the data based on the information from 502 and the addresses described above. A data sending unit 505 sends video data to the network at an appropriate speed and may include an FEC function.
[0040] The appropriate speed means keeping the packet transmission speed below a certain level and keeping the packet transmission interval constant within that rate so that packet loss in the network and delays due to buffering in the network device are not excessive. For example, even if the network bandwidth is 10 Gbps, if the video signal rate is 1 Gbps, the packet transmission rate should be kept at about 1 Gbps and the packet transmission interval should be kept constant to keep it at about 1 Gbps, thereby preventing packet loss and excessive delays in the network.
[0041] (Receiving device) A data receiving unit 601 receives packets from the network. At this time, the data receiving unit selectively receives video data transmitted using a predetermined address according to the network bandwidth of the data receiving unit or the image quality that can be displayed on the display device. Packet loss data may be restored using the FEC function. A depacketizer 602 extracts data from packets. A signal analysis unit 603 reads the data from 602 and judges whether or not the audio data, etc. have been rearranged. This may be determined from the marker data. 604 is a data adjustment unit which replaces and rewrites some data, such as audio data, based on information from 603, constructs video data that can be played on a display device, and passes the data to 605. If it is necessary to reduce the horizontal resolution, data thinning and integration processing is performed. Although thinning of the vertical resolution is possible when an implementation that changes the address for each scan line is performed, thinning of the horizontal resolution is difficult, so the horizontal thinning is performed on the receiving side. A data output unit 605 performs data buffering and IF conversion. For signals from a display device such as HDMI that require a response, the response is made by this data output unit.
[0042] In the present disclosure, the receiving device and the display device may be provided in one device. For example, in Fig. 1, the receiving unit and the display unit provided in the terminal may be used as the receiving device 102 and the display device 132. In addition, the transmitting device and the receiving device of the present disclosure may be realized by a computer and a program, and the program may be recorded on a recording medium or provided through a network.
[0043] (Effects Produced by Inventions) It is possible to provide a means for transferring a single large-capacity video image via multiple networks with different available bandwidths, and to multiple display devices in remote locations with different display capabilities.
[0044] (Key Points of the Invention) Conventionally, uncompressed video has been transmitted over short distances of several meters to several tens of meters using an interface. Although systems have been proposed that transmit this data over a network with low latency, simultaneous distribution to multiple receiving devices has only been achieved using multicast, and simultaneous distribution to display devices with low network bandwidth or different display capabilities has not been possible.
[0045] In the present invention, the transmitting side transmits one image with different identifiers for distribution depending on the type and order of data such as pixels and frames, and the receiving side receives packets selectively based on the identifiers, thereby obtaining images with the required number of pixels and frames. As a result, the transmitting side can selectively receive and display images according to the network bandwidth and displayable image quality of the receiving side by transmitting only one image. In particular, when the receiving side has multiple networks and multiple receiving devices, the present invention does not require multiple image sources, making it a system that is easily expandable. Furthermore, by using existing network packet header identifiers for traffic selection on the network, it is possible to realize the system using existing network equipment. [Industrial Applicability]
[0046] The present disclosure can be applied to the information and communications industry. [Explanation of symbols]
[0047] 101: Transmitting device 102, 103: Receiving device 112, 113: network boundary device 121, 122, 123: Network 500: Transmitting device 501, 601: Input buffer 502, 603: Signal analysis section 503, 604: Data adjustment section 504: Packetizer 505: Data transmission unit 600: Receiving device 602: Depacket Department 605: Data output section
Claims
1. a transmitting device separates the uncompressed video data into packets by at least one of frames and scanning lines, and transmits the packets by writing an address corresponding to at least one of the frames and scanning lines in a header; A receiving device selectively receives uncompressed video data in which an address determined by at least one of a network bandwidth and image quality of the receiving device is written in a header. Video distribution system.
2. The transmitting device transmits data different from the video data by describing an address different from the video data in a header. The video distribution system according to claim 1 .
3. the transmitting device packetizes first uncompressed video data obtained by thinning out at least one of frames and scanning lines from a video source, and second uncompressed video data obtained by thinning out the first uncompressed video data from the video source, and transmits the first uncompressed video data and the second uncompressed video data by describing different addresses in headers; The video distribution system according to claim 1 .
4. The receiving device includes: receiving the first uncompressed video data and the second uncompressed video data; combining the first uncompressed video data and the second uncompressed video data to construct uncompressed video data that can be played back on a display device; The video distribution system according to claim 3.
5. a transmitting device separates the uncompressed video data into packets by at least one of frames and scanning lines, and transmits the packets by writing an address corresponding to at least one of the frames and scanning lines in a header; A receiving device selectively receives uncompressed video data in which an address determined by at least one of a network bandwidth and image quality of the receiving device is written in a header. Video distribution methods.
6. a signal analyzer for identifying at least one of frames and lines of uncompressed video data; a packetizing unit that separates the uncompressed video data into packets by at least one of frames and scanning lines; a data sending unit that writes an address in a header, the address being determined by at least one of a network bandwidth and image quality corresponding to at least one of a frame and a scan line, and sends out each packet generated by the packetizing unit; A transmitting device comprising:
7. A transmission program for causing a computer to function as each of the functional units included in the transmission device according to claim 6.
Citation Information
Patent Citations
Transmission method
JP2015005976A
Packet conversion device
JP2019118076A
Method and device for providing content-related information for multimedia services
JP2019506098A
Method of transmission of digital images and reception of transport packets
US20090313669A1
