Processing and transmission of video data

Abstract

The invention relates to an integrated circuit arrangement (70), in particular an FPGA or ASIC, for processing and transmitting data, in particular video data (V1, V2, V3, V4, V5, V6, V7, V8), in a network (20). The invention also relates to a network device (50), a network (20), a method for processing and transmitting data, a computer program product and a computer readable medium.

Description

Technical Field

[0001] The present invention relates to an integrated circuit device, a network device and a network, and a method for processing and transmitting data, said data including at least video data. Background Technology

[0002] In video technology, at least the following steps are typically performed to transmit a video signal. First, the video signal is captured from a source, such as a camera or another recording device. Then, in another step, the video signal is output to a receiver, such as a monitor, display, or data storage device. However, in practice, it is often desirable not only for point-to-point connections, where one source is connected to one receiver, but also, in various applications, for the distribution of video signals from one or more sources to multiple receivers. In practice, the video signal is often also processed (composed) between the capture and output, such as zooming in, zooming out, cropping, converting to another frame rate, merging, etc.

[0003] To date, the above steps have been performed using independent or separate, dedicated devices. That is, one or more devices form the interface to the video source. One or more devices form the interface to the video receiver. At least one router is placed between the video source and the video receiver as an additional device, and other devices are placed between the source and receiver as processing units if necessary. All these devices must work together to transmit and, if necessary, process the video signal.

[0004] There is often a desire to bridge greater distances between devices, especially between source and receiver, rendering traditional interfaces (such as DP and HDMI) insufficient, as they typically have a limited range of only three to five meters. Due to the increasing demands for higher image resolutions (HD, 4K), it is challenging to transmit these video signals uncompressed and with low latency via standard networks such as Ethernet. SDVoE using 10G Ethernet, as known in practice, must already perform compression at 4K resolution, 4:4:4, and 60fps. Summary of the Invention

[0005] Therefore, the object of the present invention is to enable and / or simplify the transmission of uncompressed video signals in a network with low latency.

[0006] The objective is achieved by the integrated circuit device according to the invention, the network device according to the invention, the network according to the invention, and the method according to the invention for processing and transmitting data.

[0007] The integrated circuit device mentioned at the beginning is specifically implemented as an integrated circuit, such as an FPGA or ASIC, and is used for processing and transmitting data in a network, the data including at least video data. The integrated circuit device is specifically configured such that—if desired—data can be transmitted uncompressed. For this purpose, the integrated circuit device has a number of video interfaces for receiving and / or transmitting video data from a data stream. Furthermore, the integrated circuit device includes a number of network interfaces for connecting the integrated circuit device to a substantially identical, spatially separated number of integrated circuit devices for transmitting data, the data including at least video data. The integrated circuit device also includes a frame buffer control device and a frame buffer interface. Here, the frame buffer control device controls access to the frame buffer by means of the frame buffer interface, such that video data from one or more data streams can be stored in and / or read from the frame buffer as frames of combined video data. Data can be exchanged between the aforementioned components by means of a connection matrix of the integrated circuit device. Here, the routes of the individual data streams are defined by means of a configurable addressing scheme.

[0008] Integrated circuit devices are particularly flexible as FPGA implementations because the FPGA can be configured according to the application. Integrated circuit devices as ASIC implementations can be manufactured cost-effectively, especially in the case of large numbers of components.

[0009] In principle, the video interface can involve any video interface, such as digital interfaces like DP, DVI, HDMI, SDI, etc., or analog interfaces like VGA-Video, XGA-Video, S-Video, etc. Preferably, the video interface is configured to receive or transmit video signals uncompressed. The number of video interfaces can be configured as needed. That is, depending on the application, the integrated circuit device may, for example, include only video interfaces for receiving video signals, only video interfaces for transmitting video signals, or any combination of the video interfaces. However, at least one of the video interfaces is included by the integrated circuit device. Preferably, however, the integrated circuit device has not only a certain number of video interfaces for receiving video signals but also a certain number of video interfaces for transmitting video signals. Within the scope of this invention, the term "certain number" generally means "one or more".

[0010] Video data typically exists in the form of a data stream (also known as a video stream or video information stream), which is a sequence of images that transmit video data sequentially, frame by frame and pixel by pixel.

[0011] With the aid of a network interface, an integrated circuit device or a network device including the integrated circuit device can be connected to another integrated circuit device, particularly according to the invention. The term "substantially identical" means that the circuit devices may differ in the number and / or design of their video interfaces, network interfaces, and / or video processing units, but in other cases are designed of the same type.

[0012] The frame buffer control device controls access to the frame buffer via the frame buffer interface, so that video data from one or more data streams can be stored in and / or read from the frame buffer as frames of combined video data.

[0013] Therefore, in other words, the frame buffer control device and frame buffer interface can combine different incoming video signals into a common video signal. Thus, a frame relates to a single image, which—particularly in terms of the frame frequency on which the transmission is based—is stored in and read from the frame buffer. Therefore, the sequence of combined frames forms a combined video signal. Thus, by means of the frame buffer control device and frame buffer interface that work in conjunction with the frame buffer, individual video signals can be, particularly preferably, combined into a new combined video signal according to a user preset.

[0014] The connection matrix, especially as a multi-layered connection matrix, is designed or sized such that it can forward data from connected components to the corresponding target components independently and in parallel, particularly based on component addresses. For this purpose, the connection matrix can, for example, be constructed similarly to an AXI bus.

[0015] Integrated circuit devices are preferably used in static networks with a known topology that remains the same during operation—that is, where network devices are typically not added to or removed during operation. Correspondingly, the routes—preferably optimized—between the various network devices and their corresponding components are known, allowing addressing schemes to be created during design or network installation. Subsequently, during operation, the addressing schemes can still be configured to enable or disable individual image processing processes or video processing units. Addressing scheme selection can also be achieved by configuring which source (from which source) the video signal is transmitted to which (Senke(n)), and which route the video signal may optionally take through other network devices and their components. However, in principle, the network topology can also be changed as needed by adding and / or removing network devices; however, this then requires a new configuration of the network, and especially the addressing scheme.

[0016] The network device mentioned at the beginning has an integrated circuit device according to the present invention and a physical interface for connecting to the corresponding interface of the integrated circuit device. Therefore, the network device can connect to video sources, video receivers, and other network devices according to the interface, and preferably can also connect to additional devices, such as USB devices, computing units, or PCs, using other suitable interfaces.

[0017] In principle, the video source can be any device that outputs a video signal. The video source may include, for example, a camera, medical imaging modalities such as X-ray equipment, computed tomography (CT) scanners, magnetic resonance imaging (MRI) scanners, ultrasound equipment, etc., video signals generated by software and / or measuring devices such as EKG, EEG, etc., video signals retrieved by data storage, etc. In principle, any device that outputs a video signal can be used as a video receiver. The video receiver may include, for example, a monitor, display or other display device, data storage, a connection to an external network, etc. On the other hand, a USB device may include input devices such as a mouse, keyboard, touchpad, etc.

[0018] The aforementioned network is formed by these components, in which high-resolution video signals are transmitted, in particular. The network comprises a number of network devices interconnected in a known network topology according to the invention. As mentioned above, the network may additionally include other components if necessary. This means that the network and its topology are configured and parameterized for the corresponding application. Therefore, the network devices, their components, and their interconnections are known and taken into consideration when creating, preferably automatically optimized, addressing schemes. By means of an addressing scheme from source to receiver, a route through the network is defined for each data stream.

[0019] The method mentioned at the beginning for processing and transmitting data, including at least high-resolution video data, in a network includes the following steps, which are implemented by means of an integrated circuit device. In one step, high-resolution video data from a data stream is received and / or transmitted. In another step, the integrated circuit device is connected in the network, via a network interface, to a substantially identical, spatially separated number of integrated circuit devices for data transmission. In yet another step, video data from one or more data streams is stored in and / or read from a frame buffer as frames combining video data. Furthermore, in one method step, data is exchanged between components of the integrated circuit device by means of a connection matrix. Here, a configurable addressing scheme is used to define the routes of the individual data streams.

[0020] The order of the steps is not fixed, and the steps are typically performed in parallel during normal operation. Therefore, depending on the design, a network device may serially receive video data from one or more sources and then serially forward the video data to one or more other network devices. Alternatively, the network device may serially receive video data from the network and serially forward the video data to one or more video receivers. It is also possible to implement the two processes in parallel and / or process one or more video streams in parallel within the network device.

[0021] Most of the aforementioned components of a network device can be implemented, either entirely or partially, in the form of integrated circuits or logic devices.

[0022] However, in principle, the components can also be implemented partially, especially when particularly fast computations are involved, in the form of software-supported hardware, such as FPGAs, ASICs, etc. Similarly, the required interfaces, for example when only data reception from other software components is involved, can be configured as software interfaces. However, the interfaces can also be configured as hardware-based interfaces controlled by appropriate software.

[0023] The largely software-based implementation offers the advantage of allowing existing network devices to be easily retrofitted and updated via software modifications or FPGA reconfiguration to operate in accordance with the invention. In this regard, the objective is also achieved through a corresponding computer program product, which can be configured into the programmable logic of the network device. This computer program product has program segments or logic instructions that, when the logic is executed in the network device, perform the steps of the method according to the invention. In addition to the computer program, such a computer program product may, if necessary, include additional components, such as documentation and / or additional parts, and may also include hardware components, such as hardware keys (software dongles, etc.) for using the software.

[0024] Computer-readable media, such as memory sticks, hard disks, or other portable or fixed-mount data carriers, can be used for transport to and / or for storage in network devices, on which programmable and executable program segments containing computer programs are stored. The integrated circuit device may, for example, have one or more cooperating logic elements.

[0025] Other particularly advantageous designs and modifications of the invention can be derived from the following description, wherein embodiments of one category of embodiments may also be modified similarly to embodiments of another category of embodiments and the description portion, and features of different embodiments or variations can in particular be combined to form new embodiments or variations.

[0026] The integrated circuit device preferably includes a number of video processing units, which can be configured to process video data.

[0027] The video processing unit is preferably configured such that it can perform typical video processing procedures on a circuit device or separately on a circuit device. This is particularly true for video data streams or video streams. This is particularly true independent of other video data streams. Furthermore, typical video processing procedures include scaling, such as scaling of resolution, color space, or frame rate; local generation; filtering, such as filtering of color, noise, etc.; line reconstruction; compression or decompression. The image processing procedures and their software or hardware implementation schemes are known to those skilled in the art. The video processing unit can be configured to, for example, set a scaling factor, set the size and / or position of a local image, set filter coefficients, etc.

[0028] The video processing unit is preferably configured for scaling, local generation, filtering, line reconstruction, compression and / or decompression.

[0029] Therefore, the video processing unit is implemented using integrated circuit devices, thus operating very efficiently. Consequently, latency can be kept generally low. Here, scaling preferably includes scaling of resolution, frame rate, and / or color space. Filtering preferably includes color filters, contrast filters, edge filters, and / or noise filters. Software and / or hardware implementations of the video processing unit are generally known to those skilled in the art.

[0030] Particularly preferably, the video processing units are configured to perform exactly one of the processes described herein. That is, the video processing units are configured to perform scaling, local generation, filtering, line reconstruction, compression, or decompression. This allows for the individual application of processes to the video signal as needed. However, multiple video processing units can act sequentially on the video signal, resulting in a combination of these processes.

[0031] Preferably, the network interface has a data rate of 5 Gbit / s or higher, and more preferably 10 Gbit / s or higher. With a data rate of 5 Gbit / s, HD video signals can be transmitted uncompressed. For 4K video signals, it is preferable to relay or combine two network interfaces, each with a data rate of 10 Gbit / s, for uncompressed transmission.

[0032] The physical interfaces of network devices, especially network interfaces, are preferably designed as slots. These slots can be configured as needed, for example, using SFP modules. This allows for adjustment of network bandwidth and avoids the cost of unnecessary interfaces.

[0033] Therefore, preferably, a certain number of network interfaces connected to the same other integrated circuit device, i.e., two or four network interfaces, are relayed to provide a correspondingly higher bandwidth for the video signal to be transmitted.

[0034] Preferably, the video data includes at least high-resolution video data or video signals (according to the HD video standard). In the context of this invention, video data is understood to have an image resolution of at least 1280×720 pixels, common in HDTV, at a refresh rate or frame rate of 50 fps. However, the resolution is preferably 1920×1080 pixels. Of course, low-resolution video signals can also be processed and transmitted using integrated circuit devices.

[0035] Particularly preferably, the video data includes ultra-high resolution video data (according to the UHD video standard). "Ultra-high resolution video data" is understood to mean video data with a resolution of 4K or higher. "4K" means a resolution on the order of 4000 pixels per line, such as 3840, 4096, etc. Particularly preferably, the video data also includes video data with 8K resolution (approximately 8000 pixels per line). This is particularly advantageous to achieve detailed display even on larger display structures.

[0036] Preferably, by means of a frame buffer control device, positions are allocated in the frame buffer for the video data streams, thereby defining the placement of the video data within the frames for each corresponding data stream. Therefore, the frame buffer control device—particularly preferably based on user presets—also controls the positions within the generated combined video frames by means of the corresponding positions stored in the frame buffers for each original video signal frame. Thus, it is advantageous to combine the various video signals according to requirements.

[0037] The frame buffer is preferably configured as a multi-layer frame buffer. In a multi-layer frame buffer, multiple layers or image layers can be stored and retrieved simultaneously using a frame buffer control device and a frame buffer interface. The frame buffer control device preferably includes a layer control device, which generates new video signals to be read from the various layers, for example, by means of alpha-blending. This requires at least two image layers, such as a foreground layer and a background layer. However, other image layers, such as a layer for a mouse pointer, can also be configured in the frame buffer, as still described below.

[0038] The integrated circuit device preferably includes a mouse pointer control unit that provides an image of the mouse pointer and assigns it a defined position in a frame buffer.

[0039] Particularly preferably, the mouse pointer control unit provides a certain number of images for the mouse pointer to be selected according to the context, such as selecting a normal mouse pointer in a normal state, or selecting a loading symbol or a waiting symbol as the mouse pointer in a loading state.

[0040] Preferably, the mouse pointer position is assigned in the frame buffer based on user input, particularly mouse input. If the frame buffer is configured as a multi-layer frame buffer, the mouse pointer or its image is preferably stored at the corresponding position in the assigned image layer. The user input is transmitted to the mouse pointer control unit and subsequently to the frame buffer, for example, via an input device connected to the (agent) processor, particularly by means of a USB signal transmitted via Ethernet, as will be explained in more detail later. Thus, a display user interface can also be advantageously implemented in the combined video signal.

[0041] Alternatively or additionally, the integrated circuit device preferably has a window control unit that resets the released position as the data stream of video data moves within the frame buffer.

[0042] Therefore, the window control unit advantageously ensures that only current information is included in the combined video signal. This can be done, for example, by setting the corresponding position to a background value or setting the alpha value of a previously saved video signal to zero.

[0043] Preferably, an addressing scheme is specified for the entire network during network design or setup. The addressing scheme preferably includes internal component addresses, by means of which routes for exchanging data streams via a connection matrix are defined. Furthermore, the addressing scheme preferably includes network addresses, by means of which at least one route for exchanging corresponding data streams via the network is defined.

[0044] Before data from a data stream or packet leaves a component, the component address is set to the address of the next target component on the network device. This can be done statically for a portion of a component, where data from that component is sent sequentially only to the designated target component. However, the target component can be reconfigured as needed. Other components, such as network interfaces or Ethernet interfaces, can have their component addresses set based on network addresses or Ethernet addresses—for example, according to a lookup table. For video data streams newly introduced into the network from a video source, both the network address and the component address are set in the video interface.

[0045] Depending on the application, there is a need to transmit a video signal from a video source to multiple video receivers, or more commonly, to transmit a signal to multiple target locations and potentially output it there. Preferably, the data stream is copied according to an addressing scheme as needed to send it to multiple targets in the network. For transmission to individual video receivers, a so-called "unicast address" is provided in the addressing scheme, while for transmitting the video signal from the video source to multiple video receivers, a so-called "multicast address" is provided. The multicast address not only associates the corresponding signal with a route but also specifies the transmission tree or branch map originating from the video signal source. For this purpose, the signal is copied or multiplied, for example, in the corresponding designated network interface, and then transmitted via additional branches as necessary along the different routes specified by the addressing scheme.

[0046] In other words, the addressing scheme regulates routing within the network and, if necessary, the replication of data flows. Therefore, no additional devices, such as routers or switches, are required, as the functionality is already included in the integrated network device or its integrated circuit device according to the invention.

[0047] The integrated circuit device preferably includes an Ethernet interface for receiving and / or transmitting Ethernet data and an Ethernet control unit, wherein the Ethernet control unit transmits Ethernet data in accordance with an addressing scheme.

[0048] The addressing scheme preferably includes converting Ethernet addresses into network addresses or component addresses, for example, by converting Ethernet addresses into network addresses or component addresses in the form of an Ethernet lookup table. That is, Ethernet addresses are converted into the addressing scheme of the network according to the invention for transmission within the network according to the invention.

[0049] Here, the Ethernet data particularly preferably includes HID (Human Interface Device) data of a USB device with a known profile. In principle, although the USB standard does not propose compatibility for transmission over Ethernet due to the response time specified therein, it is feasible in known profiles of USB data—such as those for keyboard or mouse input—to simulate corresponding responses in order to control the establishment or termination of a USB connection and to avoid repeatedly sending USB packets. This step is preferably implemented by means of corresponding software running on the processor of the network device. Therefore, advantageously, HID data with a known profile can also be transmitted as Ethernet data via the network according to the invention. Thus, the network device includes the functionality of a Keyboard-Mouse-Switch.

[0050] The network device preferably includes a frame buffer in which at least one frame of video data can be temporarily stored before it is transmitted via a video interface and / or a network interface. The frame buffer is connected to an integrated circuit device via a frame buffer interface and is operated by means of a frame buffer control device as described above.

[0051] Preferably, the network device includes a processor that connects to the integrated circuit device via an Ethernet interface. Furthermore, the processor configures addressing schemes and / or parameters for image processing as needed using a UDP unit included in the integrated circuit device.

[0052] The processor, for example, processes user input and sends corresponding UDP data to the UDP unit via the Ethernet interface using an Ethernet connection. The UDP unit then converts the UDP data into a format suitable for write access to the connection matrix or other components of the network device.

[0053] The corresponding instructions for reconfiguring the addressing scheme or the parameters for image processing can be transmitted over the network as UDP data, like other Ethernet data, to the network device or the corresponding target network device.

[0054] For security reasons, it is desirable that the control or configuration of the network as described herein be performed solely by means of network devices. These network devices, referred to as proxy network devices, are connected to an external computing unit via an Ethernet interface. Thus, with the aid of the computing unit, the user provides instructions to the proxy network device for controlling the network, particularly for routing video signals and / or controlling video signal sources. The proxy network device then forwards the instructions according to the addressing scheme. Therefore, communication operating internally within the network remains hidden from the user. Furthermore, communication within the network via Ethernet data is preferably—besides the proxy network device—decoupled from the external network. Attached Figure Description

[0055] The invention will now be described in detail again with reference to the accompanying drawings and embodiments. Here, in the different drawings, the same parts are given the same reference numerals.

[0056] The accompanying diagrams are usually not to scale. The diagram shows:

[0057] Figure 1 A schematic block diagram of an embodiment of a network according to the present invention is shown.

[0058] Figure 2 A schematic block diagram of an embodiment of a network device according to the present invention is shown.

[0059] Figure 3 A schematic block diagram of one embodiment of the circuit device according to the present invention is shown.

[0060] Figure 4 This illustrates an exemplary route for a video signal. Figure 3 The circuit devices in, and

[0061] Figure 5 A schematic block diagram of two network interfaces for a circuit device according to the present invention is shown. Detailed Implementation

[0062] exist Figure 1 The diagram illustrates a network 20 according to the present invention. In this embodiment, network 20 includes six network devices 50 implemented substantially identically. However, it is readily possible to adjust the number and configuration of the network devices 50 according to requirements or application. One of the network devices is connected to a computing unit 25 or a computer 25. The network device 50 is also referred to hereinafter as a proxy network device 50. Furthermore, by means of the computing unit 25, instructions for configuring network 20 can be sent as Ethernet data E to the network device 50 via an Ethernet connection.

[0063] Two network devices in network device 50 are connected to video signal source 21. Video signal source 21 may be implemented as a camera, but in this case, it may also refer to video signals generated by software, from medical imaging modalities such as X-ray equipment, or output as video signals by measuring devices such as EKG or EEG.

[0064] Video signal source 21 transmits its video signals V1 and V2 to network devices 50 respectively connected to it. The network device 50 to which video signal V2 is sent is directly connected to the proxy network device 50. The network device 50 to which video signal V1 is sent is indirectly connected to the proxy network device 50 via another network device 50. Each video signal source 21 has a control unit 22, from which the connected network devices 50 send USB data U1 and U2 containing control commands. This allows control of settings for the video signal source 21, such as focus settings, zoom settings, and orientation settings.

[0065] Network devices 50 transmit network data N to each other. Network data N comprises specific data packets, which in particular include video signals V1, V2, V3, and V4, Ethernet data E, and USB data U1 and U2 as useful data. In addition to useful data, network data N includes transmission control data, which includes address data, checksums, etc.

[0066] The computing unit 25 is connected to a medical imaging modality 24 in the form of a computed tomography scanner 24. The computed tomography scanner 24 sends raw data R4 to the computing unit 25. In the computing unit 25, the raw data R4 is reconstructed by means of a reconstruction unit 26 and forwarded as a video signal V4 to the frame buffer 28. Additionally, a software-generated video signal V3 is sent to the frame buffer 28. The two video signals V3 and V4 are stored in two different frames F1 and F2. The difference between the two frames F1 and F2 is the scaling of the video signals V3 and V4. For example, the video signals V3 and V4 are configured to be longer in the horizontal direction in frame F1 than in frame F2. The scaling can be set by the user as needed. Therefore, the frame buffer 28 writes at the frame rate present at the video signal and may read at another frame rate, where arbitration logic can prevent possible conflicts between read and write accesses.

[0067] Video signals V3 and V4, or frames F1 and F2, are sent to proxy network device 50, wherein video signals V3 and V4 are included in frames F1 and F2. In proxy network device 50, the video signals V3 and V4 from frames F1 and F2 are merged with video signals V1 and V2 from other video signal sources 21 in frame buffer 51 to form frames F3 and F4. Corresponding to a user preset, each video signal V1, V2, V3, and V4 is scaled within frames F3 and F4.

[0068] Frames F3 and F4 are packaged into network data N and transmitted as a combined video signal to a network device 50 connected to a video signal receiver or output mechanism 23, which is in the form of a monitor, display, or similar device. The image content of frames F3 and F4 is transmitted from the network device 50 to the output mechanism 23 as combined video signals V5 and V6, where it is displayed as video. Furthermore, the network device 50 that outputs video signal V5 is connected to a USB input device 29, such as a mouse or keyboard. With the help of the USB input device 29, control data as USB data U5 can be detected, for example, and this control data is transmitted to the control unit 22 of the video signal source 21 via the network 20 as needed.

[0069] Therefore, by means of the network 20 according to the invention, video signals V1, V2, V3, V4 from different video signal sources can be detected, processed, and / or compiled as needed, transmitted over the network on different routes, and output to a video signal receiver selected by the user. The following is based on... Figure 2 The functions of network device 50 are described in detail.

[0070] Network 20 is designed according to its design scheme, thus having a known topology. Therefore, routes between network device 50, components included in network device 50, and components connected to network device 50 can be determined and recorded in an addressing scheme. Thus, by means of the addressing scheme, routes from one point in network 20 to another point in network 20 can be determined, thereby forwarding corresponding data flows as needed.

[0071] Figure 2 Exemplary and illustrative, a network device 50 according to the present invention is shown as a block diagram. The network device 50 includes an integrated circuit device 70 as a central component, said integrated circuit device 70 being implemented herein as an integrated circuit 70, particularly an FPGA. Also according to Figure 3 Detailed description of integrated circuit 70.

[0072] Network device 50 has physical connectors 52, 53, 54, and 55 for video signals. Two display port inputs 52 and two HDMI inputs 53 are used to receive video signals. Here, the display port inputs 52 and HDMI inputs 53 are each connected to a common multiplexer 61, wherein only one of the two inputs 52 and 53 is used simultaneously. Both the HDMI and display ports have four signal pins or lanes. The lanes of the used connectors 52 or 53 are aggregated onto the signal line by means of the multiplexer 61 and transmitted to the integrated circuit 70.

[0073] Conversely, integrated circuit 70 outputs the multiplexed video signal to demultiplexer 62, which outputs the aggregated video signal in a manner corresponding to the standards of the respectively connected output terminals 54 and 55, dividing the signal onto different channels. Two display port output terminals 54 and two HDMI output terminals 55 constitute a configuration for transmitting video signals, wherein the display port output terminals 54 and HDMI output terminals 55 are each connected to a common demultiplexer 62, but only one of the output terminals is used at a time.

[0074] The DisplayPort terminal here supports so-called Display-Port-Multi-Stream (DP-MST), in which at least two video signals can be transmitted via the DisplayPort terminal.

[0075] The design schemes described for video signal input terminals 52 and 53 and video signal output terminals 54 and 55 consider that one connection type often dominates at the installation location of network 20. Network device 50 should, in principle, be used in a single structural form at different locations where different connection types may be required. Other video connectors, such as VGA, S-Video, or DVI, are also compatible.

[0076] Network device 50 also has eight network terminals 63 and 64. Four of these network terminals are designed as SFP modules 63 and can be equipped with SFP transceivers as needed to save on the cost of using more expensive transceivers. The SFP transceivers can be configured as single-mode or multi-mode transceivers depending on the bridging distance, and can have transmission rates of, for example, 5 Gbit / s or 10 Gbit / s as needed. The remaining four network terminals are exemplarily designed as PCIe terminals 64, allowing network device 50 to be incorporated as a plug-in module into suitable devices, such as computing units or medical imaging modalities, when necessary.

[0077] The number of video terminals 52, 53, 54, 55 and network terminals 63, 64 of network device 50 is basically related to the processing power of integrated circuit 70, and can also be arbitrarily selected when designing network device 50.

[0078] Additionally, the network device 50 has an expansion slot 65, which can be used as a full HD output, an SPDIF audio output, for older video standards (S-Video, FBAS), as a UART or CAN terminal, or for binary input and / or output, as needed.

[0079] Furthermore, network device 50 has a frame buffer 51 in the form of one or more DDR-RAM modules, wherein the frame buffer 51 is configured as a multi-layer frame buffer. The management of frame buffer 51 is also based on... Figure 3 Describe in detail.

[0080] Furthermore, network device 50 includes a processor 59, which is connected to an Ethernet switch 60 on one side and to two USB ports 58 on the other. One of the USB ports 58 is connected to the device terminal of the processor 59, wherein the processor, by means of software emulating a USB input device, outputs USB data U1, U2 (with a known configuration file) received via network 20 to the device to be controlled. The other USB port 58 is connected to the host terminal of the processor 59 and converts USB data U3 (with a known configuration file) received by the input device into Ethernet data E, which is transmitted via network 20 as network data N.

[0081] Additionally, the Ethernet switch 60 is connected to the Ethernet terminal 57 and exchanges Ethernet data bidirectionally via it. With the help of the Ethernet switch 60, not only the Ethernet data from the Ethernet connection unit 57 but also the Ethernet data from the processor 59 are transmitted to the integrated circuit 70.

[0082] Figure 3Exemplary and illustrative, as shown by block diagram, is an integrated circuit 70 in FPGA form according to the present invention. The integrated circuit 70 includes a video module 71, an Ethernet module 72, and a network module 73. The output or input terminals of each network interface 85 are connected to the network terminals 63 of the network device 50, which is configured as an SFP transceiver. The network module 73 also... Figure 5 Detailed description.

[0083] The video module 71 has two identically designed combined video input-output interfaces 83, which are respectively connected to the multiplexer 61 and demultiplexer 62 of the network device 50. By means of the combined video input-output interfaces 83, incoming or outgoing video signals can be simultaneously forwarded in their respective directions. Incoming video signals are forwarded to either the DisplayPort-Connection Matrix-Input Interface 74 or the HDMI-Connection Matrix-Input Interface 75, depending on the standard they are based on. Outgoing video signals, corresponding to their based standard, are received via the connection matrix 93 at the DisplayPort-Connection Matrix-Output Interface 76 or the HDMI-Connection Matrix-Output Interface 77, which is defined by the component address according to the addressing scheme, and are forwarded to the corresponding demultiplexer 62 of the network device 50 via the combined video input-output interfaces 83 connected thereto.

[0084] Ethernet module 72 includes an Ethernet interface 78 and a UDP interface. Ethernet interface 78 has an Ethernet filter 81 that allows Ethernet data E to pass only to a defined network address in order to avoid irrelevant data streams.

[0085] Ethernet interface 78 also includes a buffered repeater 80 that determines the forwarding of individual packets of Ethernet data E. Outgoing Ethernet data E is forwarded to the Ethernet switch 60 of network device 50 via Ethernet filter 81, which determines whether the incoming Ethernet data E involves UDP data forwarded to UDP interface 82 or Ethernet data E forwarded on a specific route within network 20. In the latter case, Ethernet data E, based on its Ethernet address or MAC address in Ethernet connection matrix interface 79, utilizes an Ethernet lookup table to determine the network address and component address, and is forwarded accordingly via connection matrix 93.

[0086] With the help of the Ethernet interface 78, the processors 59 of the corresponding network devices 50 in the network 20 can also communicate with each other.

[0087] Integrated circuit 70 also includes frame buffer control device 87 and frame buffer interface 86. Here, frame buffer control device 87 controls access to frame buffer 51 of network device 50 by means of frame buffer interface 86, so that video data V1, V2, V3, V4, V5, V6, V7, V8 from one or more data streams can be stored in and / or read from frame buffer 51 as frames F3, F4 of combined video data.

[0088] To this end, the frame buffer control device 87 allocates positions in the frame buffer 51 for the data streams of video data V1, V2, V3, V4, V5, V6, V7, and V8, respectively, so that the video data defining the corresponding data streams is positioned in the combined frames F3 and F4. Therefore, the frame buffer control device 87 controls the position of the generated combined video frames F3 and F4 by means of the corresponding positions of the frames of each original video signal V1, V2, V3, V4, V5, V6, V7, and V8 stored in the frame buffer 51, especially according to the user's preset.

[0089] Furthermore, integrated circuit 70 includes a mouse pointer control unit 88, which provides multiple images of the mouse pointer (based on the normal mouse pointer state, loaded symbols, etc.) and assigns a defined position to it in frame buffer 51 after user input, such as user input via mouse or touchpad. The user input data can also be transmitted to the mouse pointer control unit 88 via a network, for example, as Ethernet data E.

[0090] In addition, the integrated circuit device 70 has a window control unit 89, which resets the released position when the data streams of video data V1, V2, V3, V4, V5, V6, V7, and V8 move in the frame buffer 51.

[0091] Therefore, the window control unit 89 ensures that only current information is included in the combined video signal. This can be done, for example, by setting the corresponding position to a background value or setting the alpha value of a previously saved video signal to zero. This is feasible because the frame buffer 51 is designed as a multi-layer frame buffer 51. If the alpha value of an image layer is set to zero, the layers below it are visible or displayed.

[0092] In the design of integrated circuit 70 or its basic configuration, video processing units 90, 91, and 92 can be selected according to requirements. That is, in principle, it is feasible to use not only more but also fewer video processing units. The same type of video processing unit can also exist multiple times, for example, to process multiple different video signals in a similar manner.

[0093] In the embodiment, one video processing unit is configured as a first scaling unit 90, another video processing unit is configured as a second scaling unit 91, and a video processing unit is configured as a (color) filtering unit 92.

[0094] Figure 4 Exemplary map solutions are explained in Figure 3 The routes of different video signals V7, V8, V9, and V10 in the integrated circuit 70.

[0095] The first video signal V9 and the second video signal V10 are received as Display Port Multistream (MST) and forwarded as a single data stream to the frame buffer interface 86. The frame buffer interface 86 stores the first video signal V9 and the second video signal V10 at corresponding positions in the frame buffer 51 according to the frame buffer control device 87.

[0096] The third video signal V7 is received via the first network interface 85 and forwarded to the image processing unit in the form of a first scaling unit 90 according to the addressing scheme. The first scaling unit 90 scales the size of the third video signal V7 and forwards the third video signal V7 to the frame buffer interface 86 when the component address changes. The frame buffer interface 86 stores the scaled third video signal V7' in the frame buffer 51 at the position allocated to the third video signal V7' by the frame buffer control device 87.

[0097] The fourth video signal V8 is received via the second network interface 85 and forwarded to the second scaling unit 91 according to the addressing scheme. The second scaling unit 91 scales the third video signal V7 and forwards the third video signal V7 to the (color) filtering unit 92 if the component address changes. In the color filtering unit 92, for example, blue tones are filtered out from the video signal, and then the video signal is forwarded to the frame buffer interface 86 if the component address changes. The frame buffer interface 86 stores the scaled and filtered fourth video signal V8' at the position assigned to the fourth video signal V8' in the frame buffer 51 by the frame buffer control device 87.

[0098] The frame buffer 51 can then be read in its entirety to provide a new combined video signal. This new combined video signal can then be transmitted via network 20 or output via one of the video input / output interfaces 83.

[0099] exist Figure 5 Examples are shown in the example similar to those in Figure 3 The two network modules 73 shown are identically constructed and thus described together.

[0100] Network module 73 is connected to connection matrix 93 via network connection matrix interface 101. Network module 73 receives data sent to its component addresses and sends data to other component addresses via it. Network connection matrix interface 101 has a connection matrix lookup table 102 that stores whether a route is currently active. Therefore, by configuring connection matrix lookup table 102, routes can be activated or deactivated in a simple manner without changing the addressing scheme itself.

[0101] The connection matrix interface 101 is connected to the routing unit 99 for bidirectional data transmission. The routing unit 99, based on the addressing scheme and using the routing lookup table 100, determines which internal component addresses or network interfaces 85 to send the arriving network data N to. That is, for each external network address of the device, the routing lookup table 100 specifies which network interfaces 85 to use for data transmission.

[0102] If data arriving from network 20 according to the addressing scheme is directly designated as a component for its own device or a so-called multicast address is also designated as a component for its own device, then the corresponding addressing scheme forwards or copies and forwards the data to the corresponding component address of its own device by means of routing lookup table 100.

[0103] If the data from connection matrix 93 determines a component for its own device, such as when a signal from a video source has a multicast address, then on the one hand, the corresponding signal is copied and forwarded to the internal component, and on the other hand, it is forwarded to other components or network devices 50 corresponding to the multicast address.

[0104] If it is determined that the data from connection matrix 93 is intended for another network device, then the data is forwarded via the corresponding network interface 85 using the routing lookup table 100, according to the addressing scheme.

[0105] If data arriving via network 20 is determined to be for another network device, then the data is forwarded to the corresponding network interface 85 using the routing lookup table 100 according to the addressing scheme. If, in this type of forwarding, it is necessary to send data via a network interface 85 not connected to the same network module 73, then the data is forwarded from routing unit 99 to the routing unit 99 of the other module via direct module connection 98, and from there again sent via the corresponding network interface 85 using the routing lookup table 100 according to the addressing scheme.

[0106] Furthermore, network module 73 includes a relay unit 95, also connected to routing unit 99. This allows for the relaying of data rates from two or more network interfaces 85 as needed, for example, transmitting uncompressed 4K video signals from one network device 50 to another. Relay unit 95 can, for example, relay four individual network interfaces 85 into a virtual interface with correspondingly higher data transfer rates up to 40 Gbit / s. Here, relay control unit 96 of relay unit 95 adjusts the allocation of data to each network interface 85, ensuring that data that must be recombined arrives at the target network device 50 as simultaneously as possible.

[0107] Finally, the data is converted using HSSL unit 94 and forwarded to the corresponding network device 50.

[0108] Therefore, the integrated circuit 70 according to the invention can realize the main steps of video technology, and the network device 50 according to the invention can realize the main steps of video technology. Thus, functions that previously had to be performed by different devices can now be performed by an integrated and substantially consistent device according to the invention.

[0109] Finally, it should be reiterated that the devices and methods described in detail above are merely embodiments, and these embodiments can be modified by those skilled in the art in entirely different ways without departing from the scope of the invention. Furthermore, the use of the indefinite articles “a” or “an” does not exclude the possibility that the features referred to may exist multiple times. Similarly, the term “unit” does not exclude the possibility that the unit may consist of multiple components—unless otherwise described—that may also be spatially distributed. However, network devices explicitly refer to integrated units that include all the specified components within a structural unit.

Claims

1. An integrated circuit device (70) for processing and transmitting data in a network (20), the data including at least video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10), the integrated circuit device (70) having the following components: - A number of video interfaces (74, 75, 76, 77) for receiving and / or sending video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) in a data stream. - A number of network interfaces (84, 85) for connecting the integrated circuit device (70) to a number of identical, spatially separated integrated circuit devices (70) to transmit data (N), the data (N) including at least video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10). - A frame buffer control device (87) and a frame buffer interface (86), wherein the frame buffer control device controls access to the frame buffer (51) by means of the frame buffer interface (86), such that video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) from one or more data streams can be stored in the frame buffer (51) and / or read from the frame buffer (51) as frames (F3, F4) of the combination of video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10). - A connection matrix (93) enables data exchange between the components of the integrated circuit device (70). This involves using a configurable addressing scheme to define the routes for each data stream, and - Window control unit (89) that resets the released position when the data stream of the video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) moves in the position of the frame buffer (51).

2. The integrated circuit device according to claim 1, The integrated circuit device (70) described therein is configured as an FPGA or an ASIC.

3. The integrated circuit device according to claim 1, The frame buffer (51) therein is a multi-layer frame buffer.

4. The integrated circuit device according to any one of claims 1 to 3, wherein the integrated circuit device has a number of video processing units (90, 91, 92), the video processing units (90, 91, 92) being configured to process the video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10).

5. The integrated circuit device according to claim 4, The video processing units (90, 91, 92) are configured for scaling, local generation, filtering, line reconstruction, compression and / or decompression.

6. The integrated circuit device according to any one of claims 1 to 3, The data rate of the network interfaces (84, 85) is greater than or equal to 5 Gbit / s.

7. The integrated circuit device according to claim 6, The data rate of the network interfaces (84, 85) is greater than or equal to 10 Gbit / s.

8. The integrated circuit device according to any one of claims 1 to 3, The video data mentioned therein includes at least high-resolution video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10).

9. The integrated circuit device according to claim 8, The video data mentioned includes ultra-high resolution video data.

10. The integrated circuit device according to any one of claims 1 to 3, wherein the frame buffer control device (87) allocates positions in the frame buffer (51) for the data streams of video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) such that the video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) defining the respective data streams are set in the frame.

11. The integrated circuit device according to any one of claims 1 to 3, wherein the integrated circuit device: - It has a mouse pointer control unit (88) that provides an image of the mouse pointer and assigns it a defined position in the frame buffer.

12. The integrated circuit device according to any one of claims 1 to 3, the integrated circuit device having: an Ethernet interface (78) for receiving and / or transmitting Ethernet data (E); and an Ethernet control unit (79) transmitting the Ethernet data (E) in accordance with the addressing scheme.

13. The integrated circuit device according to claim 12, The Ethernet data (E) includes HID data (U1, U2, U3) of USB devices (22, 29) with known profiles respectively.

14. The integrated circuit device according to any one of claims 1 to 3, wherein the addressing scheme includes an internal component address, by means of which a route for exchanging data streams via the connection matrix (93) is specified, and the data streams are copied according to the addressing scheme for transmission to a plurality of targets in the network (20).

15. The integrated circuit device of claim 14, wherein the addressing scheme includes a network address, by means of which at least one route is specified for exchanging corresponding data streams via the network (20).

16. The integrated circuit device of claim 14, wherein the data stream is copied according to the addressing scheme when needed for transmission to multiple targets in the network (20).

17. A network device (50) comprising: an integrated circuit device (70) according to any one of claims 1 to 16; and physical interfaces (52, 53, 54, 55, 57, 58, 63, 64) connected to corresponding interfaces (83, 78, 85) of the integrated circuit device (70).

18. The network device according to claim 17, wherein the network device has a processor (59) connected to the integrated circuit device (70) via an Ethernet switch (60) and, as needed, configures the addressing scheme and / or parameters for image processing via a UDP unit (82) of the integrated circuit device (70).

19. A network (20) having a number of network devices (50) according to claim 17 or 18, the network devices (50) being interconnected in a known network topology.

20. A method for processing and transmitting data in a network (20), the data including at least video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10), the method comprising the steps implemented by means of an integrated circuit device (70) according to any one of claims 1 to 16: - Receive and / or send video data in the data stream (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10). - Connect the integrated circuit device (70) to a number of identical, spatially separated integrated circuit devices (70) in the network to transmit data, the data including at least video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10). - Frames (F3, F4) of the combination of the video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) from one or more data streams as video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) are stored and / or retrieved in frame buffer (51). - Data is exchanged between components of the integrated circuit device using a connection matrix (93). This involves using a configurable addressing scheme to define the routes for each data stream. - The release position is reset when the data stream of the video data (V1, V2, V3, V4, V5, V6, V7, V8, V9, V10) moves in the position of the frame buffer (51).

21. The method of claim 20, wherein the frame buffer (51) is a multi-layer frame buffer.

22. A computer program product having a computer program that can be directly loaded into an integrated circuit device (70), the computer program product having program segments that, when the integrated circuit device (70) is configured by means of the computer program, perform all the steps of the method according to claim 20 or 21.

23. A computer-readable medium storing program segments programmable on an integrated circuit device (70) so as to perform all the steps of the method according to claim 20 or 21 when the integrated circuit device (70) is configured by means of the program segments during operation.

Images