System and method for scaling video stream

Abstract

Methods and systems are provided for scaling a video stream, and particularly a video stream of medical data such as medical data from an endoscope. The video stream may be scaled to match resolution requirements of the video display device. Video scaling is carried out on a single computing chip, so that video scaling can be realized, and 8.5 Gbps bandwidth limitation of a 10G network is avoided or reduced at the same time. Using a single computing chip may also reduce costs associated with using multiple computing chips. The result is a multi-view, cost-efficient solution for displaying high quality video streams while avoiding 10G Ethernet network bottlenecks.

Description

Background Art

[0001] The present disclosure relates generally to video processing, and in particular to scaling video encoded in a Software Defined Video over Ethernet (SDVoE) format.

[0002] 10G networks are capable of transmitting data and other information at up to 10 Gigabytes per second (Gbps), enabling the rapid transfer of large amounts of data. However, for uncompressed video, 10G networks are effectively limited to 8.5 Gbps bandwidth. This results in slower processing times and bandwidth bottlenecks when processing data streams that are close to or at the 8.5 Gbps limit. Summary of the invention

[0003] The above shortcomings can be addressed using the systems and methods disclosed herein. Through the systems, devices, and apparatus discussed herein, providing a processing chip capable of processing synchronized input and output data streams in SDVoE format beneficially reduces the cost associated with processing 10G data streams, improves power consumption, and reduces the number of cables and other components required to process and transmit one or more 10G data streams.

[0004] The processing chip may also include additional encoders and decoders disposed therein so that a 10G data stream may be received and scaled by a single processing chip. The scaled 10G data stream may then be output by the single processing chip and transmitted to a display where the 10G data stream may be decoded and rendered. The use of a single processing chip eliminates the need for a chain of encoders and decoders connected to a 10G network. The use of a single chip also advantageously enables the video source to be scaled to any desired resolution, thereby providing a low-cost, multi-view solution for processing and displaying 10G streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 shows a processing unit according to at least one exemplary embodiment;

[0006] Figure 2A Aspects of a system according to at least one exemplary embodiment are shown;

[0007] Figure 2B Additional aspects of a system according to at least one exemplary embodiment are shown;

[0008] Figure 3A shows a configuration for processing a 1080P input stream according to at least one exemplary embodiment;

[0009] Figure 3B shows a configuration for processing a 4K60 input stream according to at least one exemplary embodiment;

[0010] Figure 4 A decoder according to at least one example embodiment is shown; and

[0011] Figure 5 A method according to at least one exemplary embodiment is shown. DETAILED DESCRIPTION

[0012] The exemplary systems and methods of the present disclosure will be described in conjunction with video processing. However, in order to avoid unnecessarily obscuring the present disclosure, many known structures and devices are omitted from the description. This omission should not be interpreted as a limitation on the scope of the disclosure claimed. Specific details are set forth to provide an understanding of the present disclosure. However, it should be understood that the present disclosure can be practiced in various ways beyond the specific details set forth herein.

[0013] First go to Figure 1 , aspects of a computing chip 100 are shown according to at least one exemplary embodiment. The computing chip 100 may be or include a processor, a GPU, or other processing component capable of receiving, processing, and transmitting video data. The computing chip 100 includes an encoder / decoder 104. The encoder / decoder 104 may be or include one or more encoders (e.g., a device or component that generates, for example, a 10G output stream based on an HDMI input stream) and one or more decoders (e.g., a device or component that generates, for example, an HDMI output stream based on a 10G input stream), and may provide one or more functions for scaling a video data stream or any resolution, format, and / or bandwidth. For example, and as discussed in further detail below, the encoder / decoder 104 may be capable of receiving an HDMI input stream and outputting an intermediate 10G data stream, further processing the intermediate 10G stream using a 10G scaler, and outputting a 4K60 10G stream. In some embodiments, the encoder / decoder 104 may encode and decode the data stream using one or more video compression technologies, such as H.264, VP8, RV40, DS10G, etc. In one embodiment, the encoder / decoder 104 may use an SDVoE codec to encode and decode the data stream.

[0014] The computing chip 100 may receive an input data stream 108 and transmit an output data stream 112. In some embodiments, the input data stream 108 may be or include video streams having various bandwidths. For example, the input data stream 108 may be a 1080P60 data stream (e.g., a data stream having a 1920×1080 pixel resolution displayed at up to 60 frames per second when rendered to a display), and a 4K60 data stream (e.g., a data stream having a 3840×2160 pixel resolution, or a 4096×2160 pixel resolution, etc., which can be displayed at up to 60 frames per second when rendered to a display), etc. The input data stream 108 may be a high-definition multimedia interface (HDMI) data stream, which includes uncompressed data received from, for example, a video feed (e.g., an endoscope generating a video feed of a surgical treatment site). The output data stream 112 may be or include video streams having various bandwidths based on, for example, the type of data stream input into the computing chip 100. For example, when a 1080P60 HDMI stream is input into the computing chip 100, the output data stream 112 may be a data stream with up to 1080P60 video quality, and when a 4K60 HDMI stream is input into the computing chip 100, the output data stream 112 may be a data stream with up to 1440P60. In some embodiments, the input data stream 108 may be generated during a surgical procedure or surgical treatment process, or may be otherwise generated within the context of a surgical treatment environment. For example, the input data stream 108 may be generated by an endoscope used to survey, scan, or otherwise observe a surgical treatment site and / or anatomical elements of a patient.

[0015] In some embodiments, the computing chip 100 can receive a data stream that needs to be scaled. For example, the input data stream 108 can be or include a 4K60 data stream, which will use the entire 8.5Gbps bandwidth when output from the computing chip 100. This may be undesirable because the output stream may be used as a source for video synthesis or general operating room routing of data, but not as both. To solve this problem, the computing chip 100 can use a 10G scaler set in the computing chip 100. 10G scaling can include an intermediate decoder and an encoder, which scales the input data stream 108 to produce another output stream that can be used for video synthesis, which can beneficially reduce the size of the encoding and decoding equipment, improve power consumption, reduce costs, and reduce the number of cables and other components required to connect the 10G data stream to one or more systems discussed herein. In some embodiments, the first encoder of the computing chip 100 can output an intermediate output stream 116, which can be a 10G data stream based on a 4K60 input data stream that can be used for an operating room display. Then, the intermediate output stream 116 can also be passed through another decoder. The decoder may generate an intermediate input stream 120, which may be a non-10G data stream (eg, an HDMI data stream) that is then passed as input into an additional encoder. The additional encoder may then encode the data stream to generate an output data stream 112 that may be used for video compositing.

[0016] In some embodiments, the computing chip 100 can be used within the context of a surgical treatment environment, such as when a surgeon is performing a surgical operation or surgical treatment procedure. For example, the computing chip 100 can be used to encode and decode a 10G data stream associated with a raw video feed captured by an endoscope or other surgical treatment instrument during a stage of a surgical operation or surgical treatment procedure. It should be understood that the present disclosure is in no way limited to surgical operations or surgical treatment procedures, and it is possible to use the computing chip 100 within the context of any medical field or medical environment.

[0017] Figure 2A-2B 2 shows aspects of a system 200 according to at least one exemplary embodiment. The system 200 includes a processing unit 206, a 10G network 224, a decoder 228, a first display 232, a second display 236, a memory 240, a user interface 244, a network interface 248, and a database 252. In some embodiments, the system 200 may include, in addition to Figure 2A-2B For example, in some alternative embodiments, the system 200 may omit the second display 236 and have only the first display 232 as a display.

[0018] The processing unit 206 may provide processing functionality and may correspond to one or more computer processing devices. For example, the processing unit 206 may be provided as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), any other type of integrated circuit (IC) chip, a collection of IC chips, a microcontroller, a collection of microcontrollers, or a GPU, etc. As a more specific example, the processing unit 206 may be or include The Semtech AVP2000T AV processor chip, or any similar chip capable of transmitting SDVoE formatted video streams or other similar data streams. Use of the AVP2000T AV processor chip may be particularly advantageous because the AVP2000T AV processor chip includes the SDVoE format while also being capable of both synchronous input and output of 10G data streams. Information about the AVP2000T AV processor may be accessed via the Internet address “https: / / www.semtech.com / products / professional-av / blueriver / avp2000t” and is incorporated herein by reference in its entirety. As another example, the processing unit 206 may be provided as a microprocessor, a central processing unit (CPU), a GPU, or multiple microprocessors configured to execute instruction sets and / or data stored in the memory 240. The processing unit 206 enables various functions of the system 200 when executing instructions and / or data stored in the memory 240.

[0019] The processing unit 206 includes the computing chip 100 and, optionally, the computing chip 202. The computing chip 202 includes an encoder / decoder 204, an input data stream 208, an output data stream 212, an intermediate output stream 216, and an intermediate input stream 220, which may be similar or identical to the encoder / decoder 104, the input data stream 108, the output data stream 112, the intermediate output stream 116, and the intermediate input stream 120 of the computing chip 100, respectively. Notwithstanding the foregoing, the processing unit 206 may have one or more additional computing chips in addition to the computing chip shown. The additional computing chips 202 may allow the system 200 to process additional data streams, or otherwise provide bandwidth for processing multiple streams (e.g., multiple, optionally parallel 4K60 data streams). In one embodiment, computing chip 100 and computing chip 202 may be or include an AVP2000T AV processor chip that is capable of performing various encoding, scaling, and / or decoding functions to process video streams, and more specifically video streams encoded in the SDVoE format. In this embodiment, the AVP2000T AV processor chip may operate as a video processor (e.g., a 10G scaler) to generate additional processing streams that may be used for video synthesis, thereby providing the functionality of both an encoder and a decoder. Using the AVP2000T AV processor chip may advantageously reduce costs compared to, for example, using a separate encoder and decoder chain connected to a 10G network.

[0020] The 10G network 224 may be a collection of transmitters, receivers, and / or (electrical and / or optical) cables that transmit the data stream from the processing unit 206 to the decoder 228 and, by extension, to the first display 232 and the second display 236. The 10G network 224 may be or include a collection of fiber optic cables (e.g., single-mode fiber, multi-mode fiber, etc.), 10G base cables (e.g., 10GBase-SR cables, 10GBase-LR cables, laser optimized cables, etc.), and Category 6 (“Cat6”) cables, etc. (e.g., 8K fiber optic HDMI 2.1 cables) that are capable of transmitting the output data stream to the decoder 228. In some embodiments, the 10G network 224 may include one or more wireless or Wi-Fi capabilities (e.g., the ability to wirelessly communicate with other networks, the ability to transmit data to the decoder 228 or the ability to transmit data from the decoder 228 to other locations or virtual spaces, etc.). The 10G network 224 may provide a 10Gpbs transmission speed capability and / or may otherwise have sufficient bandwidth to transmit 10G data streams.

[0021] Decoder 228 may be or include a component (e.g., a processor and logic gates, etc.) capable of converting a 10G data stream into a non-10G data stream, and may provide synthesis functionality. For example, decoder 228 may receive a 10G data stream from 10G network 224, and may synthesize the 10G data stream (e.g., scale the data to ensure that the output stream matches the pixel parameters of first display 232 and / or second display 236). In some embodiments, decoder 228 may include artifact removal functionality. In some embodiments, decoder 228 may decode output data stream 112 and output data stream 212 generated by computing chip 100 and / or computing chip 202, respectively.

[0022] The first display 232 and the second display 236 may be or include a liquid crystal display (LCD), a light emitting diode (LED) display, a high definition (HD) display, or a 4K display, etc. The first display 232 and the second display 236 may be independent displays or displays integrated as part of another device, such as a smart phone, a laptop, a tablet computer, and / or a headset or head-mounted device, etc. In one embodiment, the first display 232 and the second display 236 may be monitors or other viewing devices disposed in an operating room, so that a video feed captured from a surgical procedure or surgical treatment process may be rendered to the first display 232 and / or the second display 236 for viewing by a physician.

[0023] Memory 240 may be or include a computer-readable medium having instructions executable by processing unit 206. Memory 240 may include any type of computer memory device and may be volatile or non-volatile in nature. In some embodiments, memory 240 may include a plurality of different memory devices. Non-limiting examples of memory 240 include random access memory (RAM), read-only memory (ROM), flash memory, electrically erasable programmable ROM (EEPROM), dynamic RAM (DRAM), etc. Memory 240 may include instructions that enable processing unit 206 to control various elements of system 200 and store data in, for example, database 252 and retrieve information from database 252. Memory 240 may be local to processing unit 206 (e.g., integrated with processing unit 206) and / or separate from processing unit 206.

[0024] Database 252 includes the same or similar structure as the above-described memory 240. In at least one exemplary embodiment, database 252 is included in a remote server and stores video data captured during a surgical procedure or surgical treatment process (e.g., a camera on an endoscope captures a live feed during an endoscopy).

[0025] The user interface 244 includes hardware and / or software that enables user input to the system 200 and / or any one or more components thereof. The user interface 244 may include a keyboard, a mouse, a touch-sensitive pad, a touch-sensitive button, a mechanical button, a switch, and / or other control elements for providing user input to the system 200 to enable user control of certain functions of the system 200 (e.g., enabling / allowing the synthesis of video data streams, rendering the processed video to the first display 232 and / or the second display 236, etc.). As an illustrative example only, the first display 232 may have input buttons and switches, and in addition, a keyboard or a mouse may be directly connected to the processing unit 206. All of these together constitute the user interface 244.

[0026] The network interface 248 may enable one or more components of the system 200 to communicate with each other or with components external to the system 200 by wire and / or wireless communication. These communication interfaces that allow the components of the system 200 to communicate using the network interface 248 include wired and / or wireless communication interfaces for exchanging data and control signals between each other. Examples of wired communication interfaces / connections include Ethernet connections, HDMI connections, and / or connections that comply with the PCI / PCIe standard and the SATA standard, etc. Examples of wireless interfaces / connections include Wi-Fi connections, LTE connections, connection, and / or NFC connection, etc. In some embodiments, the network interface 248 can enable the 10G network 224 to communicate with the processing unit 206, the decoder 228, the first display 232, and the second display 236. For example, the network interface 248 may include one or more network switches that enable data to be transferred from the processing unit 206 to the 10G network 224 (and vice versa). As another example, the network 224 can communicate with the first display 232 and / or the second display 236 through the network interface 248, thereby enabling a user to interact with the 10G network 224 (e.g., using a user interface 244 coupled to the first display 232 and / or the second display 236).

[0027] Figure 3A-3B The various processing capabilities of the processing unit 206 or its components according to at least one exemplary embodiment of the present disclosure are illustrated. More specifically, the computing chip 100, 202 of the processing unit 206 may include a plurality of encoders and decoders for processing data streams and synthesizing data therefrom. Figure 3AAs shown, encoder 304 (which may be similar or identical to the encoder functionality of encoder / decoder 104 or encoder / decoder 204) may receive input stream 308. Input stream 308 may be similar or identical to input data stream 108 or input data stream 208. For example, input stream 308 may be an HDMI input stream with varying bandwidth up to 1080P60. Input data stream 208 may be derived from, for example, a video stream of a surgical treatment site. For example, an endoscope or other surgical treatment imaging device may capture a video feed of a surgical treatment site (such as a target anatomical object) during a surgical procedure or surgical treatment process.

[0028] The input stream 308 can be input into the encoder 304, and the encoder 304 can create two output streams: a first output stream 312 and a second output stream 316. The first output stream 312 can be a 10G data stream with a bandwidth of up to 1080P60. The first output stream 312 can eventually be routed to the first display 232 and / or the second display 236 using the 10G network 224 and the decoder 228. The second output stream 316 can be a data stream for synthesis. For example, the second output stream 316 can be scaled to the size required for the final synthetic output. The final synthetic output can be a picture in picture, side by side, or quad synthesis, etc. More generally, the final synthetic output can be or include different combinations of rectangular video areas that are set close to each other, overlap each other, etc. The first output stream 312 and the second output stream 316 can be controlled by a processor or other controller. For example, the API server or other processing unit (eg, processing unit 206 ) may enable or disable output of first output stream 312 and / or second output stream 316 from encoder 304 .

[0029] Go to Figure 3B , illustrating aspects of processing unit 206 according to at least one exemplary embodiment. Figure 3A , encoder 304 can receive input stream 308, which can be up to 4K60 HDMI input stream. Since input stream 308 can be up to 4K60, the resulting first output stream 312 can be compressed and use all available 10G bandwidth. The first output stream 312 can be a 10G data stream routed to an operating room or other location. However, the first output stream 312 may not be available for synthesis. Therefore, the first output stream 312 can be the only stream flowing out of encoder 304.

[0030] In addition, the processing unit 206 may still be able to generate multiple streams using the 10G scaler by passing the first output stream 312 directly into the decoder 320. The decoder 320 may convert the 4K60 10G data stream into an HDMI non-10G data stream, such as a 1080P60 data stream, etc. The data stream may be output as an intermediate stream 324, which is immediately passed into another encoder 328, which performs a similar function as the encoder 304. That is, the encoder 328 converts the intermediate stream 324 into a third output stream 332 and a fourth output stream 336. The third output stream 332 may be similar or identical to the first output stream 312, and the fourth output stream 336 may be similar or identical to the second output stream 316. The fourth output stream 336 may be a downscaled 10G data stream, such as a 1440P60 data stream (e.g., a 1920×1440 pixel resolution displayed at 60 frames per second when rendered to a display), etc. The fourth output stream 336 can then be used as a composite source corresponding to the first output stream 312. Thus, using the combination of decoder 320 and encoder 328 as a 10G scaler allows processing unit 206 to convert a 4K60 data stream into two separate 10G data streams while avoiding the 8.5 Gbps bottleneck.

[0031] When there are multiple 4K60 data streams, the processing unit 206 can utilize multiple 10G scalers, where each 10G scaler is used to process a separate 4K60 data stream. For example, if there are two 4K60 data streams, the processing unit 206 will utilize two 10G scalers. In another example, if there are four 4K60 data streams, the processing unit 206 will utilize four 10G scalers. In some embodiments, the number of streams available for synthesis can provide a limit on the number of 10G scalers. For example, if the synthesizer only uses a maximum of four streams at a time, only four 10G scalers will be required to process the 4K60 stream.

[0032] Go to Figure 4, a decoder 404 according to at least one exemplary embodiment is shown. In some embodiments, the decoder 404 may be similar or identical to the decoder 228 or the decoder 320 and may be disposed within the computing chip 100. The decoder 404 may receive one or more scaled 10G streams (such as a 1080P60 stream, a 1080P30 stream, a 720P60 stream) as a first input stream 408 and generate a first output stream 412. The first output stream 412 may be or include a non-10G data stream, such as an HDMI 2.0 output that may be displayed on, for example, the first display 232 and / or the second display 236. The parameters of the output stream may vary based on the output window mode of the display on which the video is to be displayed. In one example, the decoder 404 may be configured for a picture-in-picture (PiP) display. The PiP display may have an output resolution of 3840×2160, with a large window picture resolution of 3840×2160 and a PiP window picture resolution of 1280×720. Alternatively, the output resolution can be 1920×1080, with a large window picture resolution of 1920×1080 and a PiP window picture resolution of 640×360. Thus, the first input stream 408 can include two separate streams: a first stream with a bandwidth of up to 1080P60 and a second stream with a bandwidth of up to 720P60. The decoder 404 can then decode the streams into non-10G streams and route the first output stream 412 to the first display 232 and / or the second display 236 to be displayed. The entire composite image can be upscaled to 4K60 on the monitor of the first display 232 or the second display 236.

[0033] In another example, the decoder 404 can be configured for Picture-and-Picture (PaP) display. For example, when the output resolution is 3840×2160, both the left picture and the right picture can have a 1920×1080 resolution. Similarly, when the output resolution is 1920×1080, both the left picture and the right picture can have a 960×540 resolution. Where PaP is required, the first input stream 408 can include two 1080P60 data streams input into the decoder 404. In other embodiments, the first input stream 408 can include a data stream that allows a 2560×1440P window and a 1280×720P window to be rendered on the same display. The decoder 404 can receive two 1080P60 data streams and output a first output stream 412, which can be an HDMI 2.0 output routed to the first display 232 and / or the second display 236.

[0034] In another example, the decoder 404 can be configured to enable a Quad View (e.g., four 1080P windows) display. When the output resolution is 3840×2160, the four windows can each have a 1920×1080 resolution, and when the output resolution is 1920×1080, the four windows can each have a 960×450 resolution. In this embodiment using a quad view format, there may be four different input streams sent to the decoder 404, each of which is up to 1080P30. Since four data streams with 1080P60 will exceed the bandwidth of the 10G network, the input stream can be reduced to 1080P30 by an encoder (such as the encoder 304 of the computing chip 100) before being input to the decoder 404. Once the four data streams have been routed to the decoder 404, the decoder 404 can convert the stream to a non-10G stream, such as an HDMI 2.0 output that can be displayed on the first display 232 and / or the second display 236. Since the window display can be quad-view, all four data streams can be displayed on the same display.

[0035] For illustrative purposes only, the following is an example of a method for processing data generated in the context of a surgical treatment environment. Input data can be created in a surgical treatment environment during a stage of a surgical operation or surgical treatment process. The input data can be generated by an endoscope or other surgical treatment instrument that captures raw video of a surgical treatment site and / or anatomical elements during a surgical operation or surgical treatment process. An example of a raw video (e.g., an uncompressed video feed) can be a video associated with a surgical treatment site. The raw video can be a 10G video stream that is passed to a computer chip or other processor, and the chip or other processor can include a 10G encoder and decoder. In some cases, the computer chip or other processor can be or include an AVP2000T AV processor chip that can provide both encoding and decoding of the SDVoE format. The raw video can be processed by the encoder of the processor chip to produce an intermediate encoded data stream. Then, the encoded data stream can be passed through the decoder of the processor chip to produce a decoded stream. The decoded stream can then be passed through the encoder again to generate an output data stream. The output data stream can then be sent over a network and then decoded to produce a video stream. The video stream may be rendered to one or more displays, such as a display within an operating room, etc. To provide a visual depiction of a surgical treatment site, or for any other reason, rendering to a display may be performed so that one or more users in the treatment environment (e.g., a surgeon, a member of a surgical treatment staff, etc.) can view the video feed generated by an endoscope or other surgical treatment instrument. By providing additional encoding and decoding on a processor chip (which may be or include an AVP2000T AV processor chip), processor resources associated with processing 10G data streams may be beneficially conserved, and the overall cost associated with transmitting, processing, and rendering 10G data streams may be beneficially reduced.

[0036] Figure 5 A method 500 is shown in accordance with at least one exemplary embodiment of the present disclosure. For example, the method 500 can be used to transmit a video feed from a surgical treatment site to a display.

[0037] Method 500 includes capturing a video feed of a surgical treatment site (step 504). The video feed may be generated from a medical instrument or device viewing the surgical treatment site during a surgical procedure or surgical treatment task. For example, an endoscope may capture a live feed of a portion of a patient's esophagus during an assessment of the patient's upper esophagus. The captured feed may be routed through the endoscope to one or more components of system 200, such as processing unit 206, memory 240, and / or database 252. In some embodiments, more than one medical instrument and / or more than one video feed may be captured synchronously.

[0038] The method 500 also includes receiving an input data stream from a video feed (step 508). The input data stream may be similar or identical to the input data stream 108, the input data stream 208, and / or the input stream 308. The bandwidth of the input data stream may vary based on, for example, the quality or type of the device capturing the video feed. Non-limiting examples of the input data stream include a 1080P60 data stream, a 4K60 data stream, and a 720P60 data stream, etc. In addition, the input data stream may be a non-10G uncompressed data stream having a bandwidth less than 8.5 Gbps, so that the input data stream may be converted into a data stream capable of traversing a 10G network.

[0039] Method 500 also includes passing the input data stream through a first encoder to generate a first data stream (step 512). The first encoder can be similar or identical to encoder 304. In other words, the first encoder can receive the input data stream and create two output data streams using it. The first data stream can be similar or identical to the first output stream 312, and the second output stream from the encoder can be similar or identical to the second output stream 316. The first data stream can be a routable 10G stream (e.g., a data stream up to 1080P60) that can be routed to a display, and the second data stream can be a 10G stream for synthesis. That is, the second data stream can be scaled to the size required for the final synthesized output.

[0040] Method 500 also includes passing the first data stream through a first decoder to generate a second data stream (step 516). In some embodiments, the input data stream can be a compressed 4K60 data stream, and when output by the first encoder, all available bandwidth on the 10G network will be used. As a result, when the 4K60 data stream is passed through the first encoder, only the first data stream (e.g., a data stream that can be routed through a 10G network) is generated, and the second data stream for synthesis is not generated. In order to generate a similar second stream, the first stream can be routed through a 10G network, and the first stream is also passed to the first decoder. The first decoder can be similar or identical to decoder 228 or decoder 320, and can receive a 10G data stream from the first encoder and generate a second data stream. The second data stream can be a non-10G data stream, such as an HDMI 2.0 data stream.

[0041] Method 500 also includes passing the second data stream through a second encoder to generate an output data stream (step 520). The second data stream can be passed to a second encoder, which can be similar or identical to encoder 328. Then, the second encoder can generate two output data streams. The first output data stream can be a scaled stream that can be used for video synthesis. When the data stream is rendered to a display, the second output stream can be omitted or not used.

[0042] Method 500 also includes transmitting an output data stream on a network and through a second decoder to generate a video stream (step 524). The output data stream can be transmitted on a 10G network (such as 10G network 224). The second decoder can be similar or identical to decoder 404, and can receive a first output data stream and a first data stream output from a first encoder, and can use these data streams to generate an output data stream. The output data stream can be or include a scaled video that can be rendered to a display. In some embodiments, the second decoder can receive up to four different data streams and perform video synthesis to scale, adjust, transform or otherwise change the input stream to match the display requirements. For example, in a PiP or PaP display, the second encoder can receive two data streams of a left source and a right source constituting a picture to be displayed on the screen from two different computing chips (e.g., from computing chip 100 and computing chip 202). Then, the second decoder can adjust the stream and generate an output stream. The output stream can be a non-10G data stream, such as an HDMI 2.0 output with a bandwidth of up to 4K60. In another example, such as when the display is a quad-view display, the second decoder may receive four scaled streams up to 1080P30 from one or more computing chips and generate an HDMI 2.0 output stream up to 4K60 bandwidth.

[0043] The method 500 also includes rendering the video feed to a display (step 528). The output stream from the second decoder can then be rendered to the display. The display can be similar or identical to the first display 232 and / or the second display 236. In some embodiments, steps 504 to 524 can be continuously repeated as the video feed of the surgical treatment site is updated. For example, the physician may move the device generating the video, and the method 500 can continue so that the video stream is processed and rendered to the display in real time or near real time (e.g., a delay of one or two frames).

[0044] Any of the steps, functions and operations discussed herein may be performed continuously and automatically.

[0045] Although the exemplary embodiments described herein show various components of the system being collocated, certain components of the system may be remotely located at a remote portion of a distributed network (such as a LAN and / or the Internet, etc.), or located within a dedicated system. Therefore, it should be understood that the components of the system may be combined into one or more devices (such as a server, a communication device, etc.), or collocated at specific nodes of a distributed network (such as an analog and / or digital telecommunications network, a packet switching network, or a circuit switching network, etc.). It can be understood from the foregoing description, and for reasons of computational efficiency, the components of the system may be arranged at any location within the distributed network of components without affecting the operation of the system.

[0046] In addition, it should be understood that the various links connecting the elements may be wired or wireless links, or any combination thereof, or any other known or later developed element capable of supplying / communicating data to and / or from the connected elements. These wired or wireless links may also be secure links, and may be capable of communicating encrypted information. For example, the transmission medium used as a link may be any suitable carrier for electrical signals, including coaxial cables, copper wires, and optical fibers, and may take the form of sound or light waves, such as those generated during radio wave and infrared data communications, etc.

[0047] While the flow diagrams have been discussed and illustrated with respect to a particular sequence of events, it should be understood that changes, additions, and omissions to this sequence may occur without materially affecting the operation of the disclosed embodiments, configurations, and aspects.

[0048] Many variations and modifications of the disclosure may be used.Some features of the disclosure may be provided without providing other features.

[0049] In yet another embodiment, the system and method of the present disclosure can be implemented in conjunction with a special-purpose computer, a programmed microprocessor or microcontroller and peripheral integrated circuit elements, an ASIC or other integrated circuit, a digital signal processor, a hard-wired electronic or logic circuit such as a discrete component circuit, a programmable logic device or gate array such as a PLD, PLA, FPGA, PAL, a special-purpose computer, or any equivalent component. Generally, any device or component capable of implementing the methods shown herein can be used to implement various aspects of the present disclosure. Exemplary hardware that can be used for the present disclosure includes computers, handheld devices, phones (e.g., cellular, Internet-enabled, digital, analog, hybrid, etc.) and other hardware known in the art. Some of these devices include processors (e.g., single or multiple microprocessors), memory, non-volatile storage, input devices, and output devices. In addition, alternative software implementations including but not limited to distributed processing or component / object distributed processing, parallel processing, or virtual machine processing can also be constructed to implement the methods described herein.

[0050] In yet another embodiment, the disclosed method can be readily implemented in conjunction with software using an object or object-oriented software development environment that provides portable source code that can be used on a variety of computer or workstation platforms. Alternatively, the disclosed system can be implemented partially or completely in hardware using standard logic circuits or VLSI designs. Whether software or hardware is used to implement a system according to the present disclosure depends on the speed and / or efficiency requirements of the system, the specific functionality, and the specific software or hardware system or microprocessor or microcomputer system being used.

[0051] In yet another embodiment, the disclosed method may be implemented in part in software, which may be stored on a storage medium and executed on a programmed general-purpose computer in cooperation with a controller and memory, a special-purpose computer, or a microprocessor, etc. In these examples, the disclosed system and method may be implemented as a program (such as an applet, or CGI scripts, etc.), resources resident on a server or computer workstation, routines embedded in a dedicated measurement system, or system components, etc. The system and / or method may also be implemented by physically incorporating it into a software and / or hardware system.

[0052] Although the present disclosure describes the components and functions implemented in the embodiments with reference to specific standards and protocols, the present disclosure is not limited to such standards and protocols. Other similar standards and protocols not mentioned herein exist and are considered to be included in the present disclosure. In addition, the standards and protocols mentioned herein and other similar standards and protocols not mentioned herein are occasionally replaced by faster or more effective equivalents with fundamentally the same functions. Such alternative standards and protocols with the same functions are considered to be equivalents included in the present disclosure.

[0053] In various embodiments, configurations, and aspects, the present disclosure includes components, methods, processes, systems, and / or devices substantially as depicted and described herein, including various embodiments, subcombinations, and subsets thereof. After understanding the present disclosure, those skilled in the art will understand how to make and use the systems and methods disclosed herein. In various embodiments, configurations, and aspects, the present disclosure includes providing devices and processes in the absence of items not depicted and / or described herein or in its various embodiments, configurations, or aspects (including in the absence of such items that may have been used in previous devices or processes), such as for improving performance, ease of implementation, and / or reducing implementation costs.

[0054] The foregoing discussion of the present disclosure has been presented for the purpose of illustration and description. The foregoing is not intended to limit the present disclosure to one or more than one form disclosed herein. For example, in the foregoing detailed description, various features of the present disclosure are grouped together in one or more than one embodiment, configuration or aspect for the purpose of simplifying the present disclosure. The features of the embodiments, configurations or aspects of the present disclosure may be combined in alternative embodiments, configurations or aspects other than those discussed above. The method of the present disclosure should not be interpreted as reflecting the intention that the disclosure required for protection requires more features than the features explicitly recited in the individual claims. On the contrary, as reflected in the appended claims, the creative aspect lies in less than all the features of a single aforementioned disclosed embodiment, configuration or aspect. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim is independently regarded as a separate preferred embodiment of the present disclosure.

[0055] In addition, although the description of the present disclosure has included descriptions of one or more than one embodiment, configuration or aspect and certain variations and modifications, other variations, combinations and modifications are within the scope of the present disclosure after understanding the present disclosure, such as within the skills and knowledge of those skilled in the art. It is intended to obtain rights that include alternative embodiments, configurations or aspects (including alternative, interchangeable and / or equivalent structures, functions, scopes or steps for those claimed, whether or not such alternative, interchangeable and / or equivalent structures, functions, scopes or steps are disclosed herein) to the extent permitted, and it is not intended to disclose any exclusive patentable subject matter.

[0056] Example aspects of the present disclosure include:

[0057] According to at least one embodiment of the present disclosure, a system for processing data generated in a medical environment includes: a processor configured to receive an input stream, scale the input stream, and generate an output stream; and a network configured to transmit the output stream to a decoder.

[0058] Any aspect herein, wherein scaling the input stream comprises: passing the input stream through a first encoder to produce a first stream; passing the first stream through a first decoder to produce a second stream; and passing the second stream through a second encoder to produce the output stream.

[0059] Any aspect herein, wherein the input stream is a non-10G data stream, and wherein the output stream is a 10G data stream.

[0060] Any aspect herein, wherein the decoder receives the output stream and transmits the data stream to a display.

[0061] Any aspect herein, wherein the data stream is rendered on the display in a picture-in-picture (PiP) format, a picture-and-picture (PaP) format, or a quad-view format.

[0062] Any aspect herein, wherein the input stream is a 1080P60 stream or a 4K60 stream, and wherein the input stream is generated by a surgical instrument.

[0063] Any aspect herein, wherein the input stream is an uncompressed video feed of the surgical treatment site.

[0064] According to at least one embodiment of the present disclosure, a system for processing data generated in a medical environment includes: a processor; and a memory having data stored thereon, which, when processed by the processor, causes the processor to: receive an input stream; scale the input stream; and generate an output stream.

[0065] Any aspect herein, wherein the data further causes the processor to: forward the output stream to a network.

[0066] Any aspect herein, wherein the data further causes the processor to: pass the input stream to a first encoder to produce a first stream; pass the first stream to a first decoder to produce a second stream; and pass the second stream to a second encoder to produce the output stream.

[0067] Any aspect herein, wherein the input stream and the second stream are non-10G data streams, and wherein the first stream and the output stream are 10G data streams.

[0068] Any aspect herein, wherein the network sends the output stream to a decoder, and wherein the decoder decodes the output stream into a video stream.

[0069] Any aspect herein, wherein the video stream is routed to a display, and wherein the video stream is displayed on the display in a picture-in-picture (PiP) format, a picture-and-picture (PaP) format, or a quad-view format.

[0070] Any aspect herein, wherein the input stream comprises video data associated with anatomical elements, and wherein the input stream is generated by an endoscope.

[0071] Any aspect herein, wherein the input stream is an uncompressed video feed of the surgical treatment site.

[0072] A method for processing data generated in a medical environment, the method comprising: receiving a non-10G input data stream; scaling the non-10G input data stream by passing the non-10G input data stream through a first encoder and through a first decoder; generating a 10G output stream; and transmitting the 10G output stream to a second decoder.

[0073] Any aspect herein, wherein the scaling further comprises: passing the non-10G input data stream through the first encoder to produce a first 10G data stream; passing the first 10G data stream through the first decoder to produce a first non-10G data stream; and passing the first non-10G data stream through a second encoder to produce the 10G output stream.

[0074] Any aspect herein, wherein the 10G output stream is delivered to the second decoder over a 10G network.

[0075] Any aspect herein, wherein the second decoder decodes the 10G output stream into a non-10G video stream.

[0076] Any aspect herein, wherein the non-10G video stream is routed to a display, and wherein the non-10G video stream is displayed in a picture-in-picture (PiP) format, a picture-and-picture (PaP) format, or a quad-view format.

[0077] The phrases "at least one," "one or more than one," "or," and "and / or" are open-ended expressions of both conjunction and disjunction in operation. For example, the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more than one of A, B, and C," "one or more than one of A, B, or C," "A, B and / or C," and "A, B, or C" each mean A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.

[0078] The term "a" or "an" entity refers to one or more than one of that entity. Thus, the terms "a" (or "an"), "one or more than one," and "at least one" can be used interchangeably herein. It should also be noted that the terms "including," "comprising," and "having" can be used interchangeably.

[0079] As used herein, the term "automatic" and variations thereof means that when a process or operation is performed, it is typically completed continuously or semi-continuously without significant human input. However, a process or operation may be automatic even if the performance of the process or operation uses significant or non-significant human input if input is received before the process or operation is performed. Human input is considered significant if it affects how the process or operation will be performed. Human input that consents to the performance of the process or operation is not considered "significant."

[0080] Aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software (including firmware, resident software, microcode, etc.) embodiment, or an embodiment combining software and hardware aspects, both of which may be generally referred to herein as "circuits," "modules," or "systems." Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.

[0081] A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) would include the following: an electrical connection having one or more conductors, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0082] Computer readable signal media may include propagated data signals embodying computer readable program code, for example, in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. Computer readable signal media may be any computer readable medium that is not a computer readable storage medium and can communicate, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or apparatus. The program code embodied in a computer readable medium may be transmitted using any suitable medium (including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing).

[0083] As used herein, the terms "determine," "calculate," "compute," and variations thereof are used interchangeably and include any type of methodology, process, mathematical operation, or technique.

Claims

1. A system for processing data generated in a medical environment, the system comprising: a processor configured to receive an input stream, scale the input stream, and generate an output stream; as well as A network is configured to transmit the output stream to a decoder.

2. The system according to claim 1, wherein: Scaling the input stream comprises: passing the input stream through a first encoder to produce a first stream; passing the first stream through a first decoder to produce a second stream; and The second stream is passed through a second encoder to produce the output stream.

3. The system according to claim 2, wherein: The input stream is a non-10G data stream, and wherein the output stream is a 10G data stream.

4. The system according to claim 3, wherein: The decoder receives the output stream and transmits the data stream to a display.

5. The system according to claim 4, wherein: The data stream is rendered on the display in a picture-in-picture format (PiP format), a picture-and-picture format (PaP format), or a quad-view format.

6. The system according to claim 1, wherein: The input stream is a 1080P60 stream or a 4K60 stream, and wherein the input stream is generated by a surgical treatment instrument.

7. The system according to claim 1, wherein: The input stream is an uncompressed video feed of the surgical site.

8. A system for processing data generated in a medical environment, the system comprising: processor; as well as a memory having data stored thereon, which, when processed by the processor, causes the processor to: Receive input stream; scaling the input stream; as well as Generates an output stream.

9. The system according to claim 8, wherein: The data also enables the processor to: The output stream is forwarded to a network.

10. The system according to claim 8, wherein: The data also enables the processor to: Passing the input stream to a first encoder to generate a first stream; passing the first stream to a first decoder to produce a second stream; as well as The second stream is passed to a second encoder to produce the output stream.

11. The system according to claim 10, wherein: The input stream and the second stream are non-10G data streams, and wherein the first stream and the output stream are 10G data streams.

12. The system according to claim 8, wherein: The network sends the output stream to a decoder, and wherein the decoder decodes the output stream into a video stream.

13. The system according to claim 12, wherein: The video stream is routed to a display, and wherein the video stream is displayed on the display in a picture-in-picture format (PiP format), a picture-and-picture format (PaP format), or a quad-view format.

14. The system according to claim 8, wherein: The input stream includes video data associated with anatomical elements, and wherein the input stream is generated by an endoscope.

15. The system according to claim 8, wherein: The input stream is an uncompressed video feed of the surgical site.

16. A method for processing data generated in a medical environment, the method comprising: Receive non-10G input data stream; scaling the non-10G input data stream by passing the non-10G input data stream through a first encoder and through a first decoder; Generate 10G output stream; and The 10G output stream is transmitted to a second decoder.

17. The method according to claim 16, wherein: The scaling also includes: Passing the non-10G input data stream through the first encoder to generate a first 10G data stream; passing the first 10G data stream through the first decoder to generate a first non-10G data stream; and The first non-10G data stream is passed through a second encoder to generate the 10G output stream.

18. The method according to claim 17, wherein: The 10G output stream is delivered to the second decoder through a 10G network.

19. The method according to claim 18, wherein: The second decoder decodes the 10G output stream into a non-10G video stream.

20. The method according to claim 19, wherein: The non-10G video stream is routed to a display, and wherein the non-10G video stream is displayed in a picture-in-picture (PiP) format, a picture-and-picture (PaP) format, or a quad-view format.

Images