Apparatus and method for automatically converting video protocol in real time

Abstract

The present disclosure relates to an apparatus for converting a video protocol in real time, and the apparatus can perform conversion between different video protocols, resolve asynchronous problems caused by the difference between clock speeds of two interfaces, and convert image data into single serial image data when the image data is output in parallel based on a transmitter video communication protocol.

Description

Real-time automatic video protocol conversion device and method

[0001] The present disclosure relates to a device for automatically converting an asynchronous video communication protocol in real time.

[0002] Since the video communication protocol that transmits the image data output from the MIPI camera and the video communication protocol that receives and processes this image data are different, a design for a hardware device that converts them in the middle is required.

[0003] However, the technology that analyzes and identifies different video communication protocols in real time between the transmitter, which is a video recording device, and the receiver, which is an image processing device, and converts them has not been disclosed at present.

[0004] The purpose of the embodiment disclosed in the present disclosure is to provide a real-time video protocol conversion device.

[0005] Additionally, the embodiments disclosed in the present disclosure seek to provide a device capable of converting different video protocols.

[0006] In addition, the embodiment disclosed in the present disclosure seeks to solve the asynchrony problem caused by the difference in Clock Speed ​​of the two interfaces.

[0007] In addition, the embodiment disclosed in the present disclosure can convert image data into serial single image data when output in parallel in a transmitting video communication protocol.

[0008] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] According to one embodiment of the present disclosure for solving the above-described problem, a real-time video protocol conversion device includes: a communication unit; a storage unit storing at least one instruction; and a processor, wherein the processor executes the at least one instruction to detect, when the start of a new video line of a first protocol video received through the communication unit is detected, detect an edge of a vertical synchronization signal (Vsync) and a horizontal synchronization signal (Hsync) and a data section (Data Enable) of the first protocol video, measure values ​​of a plurality of first timing parameters based on the detected result, set a second timing parameter for video conversion based on the measured value of the first timing parameter, and convert the first protocol video into a second protocol video based on the set second timing parameter.

[0010] In addition, the processor receives the first protocol image from a first device that captures an image, provides the second protocol image to a second device that is an image processing device, and can resolve an asynchrony problem caused by a difference in Clock Speed ​​between the first device and the second device by using an Asynchronous FIFO and a Synchronizer.

[0011] Additionally, when multiple image data are output from the first device, the processor can convert the multiple image data into serial single image data and transmit the converted data to the second device.

[0012] In addition, the storage unit stores differences between the first protocol and the second protocol, and the processor detects a positive edge of a vertical synchronization signal (Vsync) and a negative edge of a horizontal synchronization signal (Hsync) of a first protocol image using an edge detector, and measures values ​​for the plurality of first timing parameters based on a result detected through the edge detector and a data section (Data Enable).

[0013] In addition, in the first protocol video, when the frame starts, a vertical synchronization signal and a horizontal synchronization signal may be generated simultaneously, and in the second protocol video, when the frame starts, a horizontal synchronization signal may be generated after the vertical synchronization signal.

[0014] Additionally, the first protocol image may have a data section generated after a horizontal synchronization signal is generated, and the second protocol image may have a data section of the same length as the horizontal synchronization signal.

[0015] In addition, the processor can measure the length of the Hsize (entire horizontal section), the horizontal response period (HSA, Horizontal Sync Active), the first horizontal porch period (HBP, Horizontal Back Porch), the horizontal active section (HACT, Horizontal Active), and the PIXEL_CNT (number of valid pixels) section using the edge detector, and calculate the HFP (Horizontal Front Porch) based on the measured results.

[0016] In addition, the processor can calculate HFP (Horizontal Front Porch) based on the following mathematical expression 1.

[0017] [Mathematical Formula 1]

[0018]

[0019] In addition, a real-time video protocol conversion method according to an embodiment of the present disclosure for solving the above-described problem is a method performed by a video protocol conversion device, comprising: a step of detecting the start of a new video line of a first protocol image received through a communication unit; a step of detecting an edge and a data section (Data Enable) of a vertical synchronization signal (Vsync) and a horizontal synchronization signal (Hsync) of the first protocol image; a step of measuring values ​​for a plurality of first timing parameters based on the detection result; a step of setting a second timing parameter for image conversion based on the measured values ​​of the first timing parameters; and a step of converting the first protocol image into a second protocol image based on the set second timing parameter.

[0020] In addition, a computer program stored in a computer-readable recording medium for executing the present disclosure may be further provided.

[0021] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0022] According to the above-described problem solving means of the present disclosure, an effect of providing a real-time video protocol conversion device is provided.

[0023] In addition, according to the above-described problem solving means of the present disclosure, different video protocols can be converted.

[0024] In addition, according to the aforementioned problem solving means of the present disclosure, an asynchrony problem caused by a difference in Clock Speed ​​of two interfaces can be solved.

[0025] In addition, according to the above-described problem solving means of the present disclosure, when output in parallel in a transmitting unit video communication protocol, image data can be converted into serial single image data.

[0026] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0027] FIG. 1 and FIG. 2 are schematic diagrams of a real-time video protocol conversion system according to an embodiment of the present disclosure.

[0028] FIG. 3 is a block diagram of a real-time video protocol conversion device according to an embodiment of the present disclosure.

[0029] Figure 4 is an example diagram for DMT Video Timing Parameter Definitions.

[0030] Figure 5 is an example diagram for Negative H & Positive V Syncs in DMT Video Timing Parameter Definitions.

[0031] Figure 6 is a drawing illustrating a DVP (Digital Video Port) Interface corresponding to the second device.

[0032] FIG. 7 is a diagram illustrating an analysis of protocol differences between the first device of FIG. 5 and the second device of FIG. 7.

[0033] FIG. 8 is a flowchart of a real-time video protocol conversion method according to an embodiment of the present disclosure.

[0034] Figure 9 is a diagram illustrating an edge detector.

[0035] Figure 10 is a diagram illustrating an HSA measurement module.

[0036] Figure 11 is a diagram illustrating an HBP measurement module.

[0037] Figure 12 is a diagram illustrating a PIXEL CNT measurement module.

[0038] Figure 13 is a diagram illustrating the HSIZE measurement module.

[0039] Figure 14 is a diagram illustrating an HFP measurement module.

[0040] Fig. 15 is a block diagram illustrating a part of the configuration of a video protocol conversion device.

[0041] Figure 16 is a diagram illustrating the operation process of a video protocol conversion device.

[0042] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0043] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0044] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0045] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0046] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0047] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0048] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0049] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0050] In this specification, the term "video protocol conversion device according to the present disclosure" encompasses various devices capable of performing computational processing and providing results to a user. For example, the video protocol conversion device according to the present disclosure may include a computer, a server device, and a portable terminal, or may be any one of them.

[0051] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0052] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0053] The above portable terminal may include, for example, all kinds of handheld-based wireless communication devices such as PCS, GSM, PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smart phones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMD).

[0054] The artificial intelligence-related functions according to the present disclosure are operated through a processor and a storage unit. The processor may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU or a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the storage unit. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0055] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating the predefined operation rules or artificial intelligence models set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0056] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weights, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network.

[0057] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that mimics human neurons (biological neurons) to enable machines to learn. Artificial intelligence methodologies can be categorized into supervised learning, where input and output data are provided together as training data, thereby determining the solution (output data) to a problem (input data); unsupervised learning, where only input data is provided without output data, so that the solution (output data) to a problem (input data) is not determined; and reinforcement learning, where a reward is provided from an external environment each time an action is taken in the current state, and learning proceeds in a direction that maximizes this reward. Furthermore, artificial intelligence methodologies can be categorized by the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks, recurrent neural networks, transformers, and generative adversarial networks.

[0058] The device may include an artificial intelligence model. The artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired outcome from an arbitrary input by changing the weights of the neurons through learning.

[0059] The processor can create a neural network, train (or learn) a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain the neural network. The models of the neural network can include various types of models such as CNN, R-CNN, RPN, RNN, S-DNN, S-SDNN, Deconvolution Network, DBN, RBM, Fully Convolutional Network, LSTM Network, Classification Network, etc., such as GoogleNet, AlexNet, VGG Network, etc., but are not limited thereto. The processor can include one or more processors for performing calculations according to the models of the neural network. For example, the neural network can include a deep neural network.

[0060] Neural networks include CNN, RNN, perceptron, multilayer perceptron, Feed Forward (FF), Radial Basis Network (RBF), Deep Feed Forward (DFF), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Auto Encoder (AE), Variational Auto Encoder (VAE), Denoising Auto Encoder (DAE), Sparse Auto Encoder (SAE), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning) Machine), ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Network) It will be understood by those skilled in the art that the neural network may include any neural network, including but not limited to a Neural Computer (NN), a Neural Turning Machine (NTM), a Capsule Network (CN), a Kohonen Network (KN), and an Attention Network (AN).

[0061] According to an exemplary embodiment of the present disclosure, the processor may be configured to perform a process for generating a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, Region with Convolution Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restrcted Boltzman Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for natural language processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet for data intelligence, Anomaly Detection, Prediction, Time-Series Forecasting, Various artificial intelligence structures and algorithms, including optimization, recommendation, and data creation, can be utilized, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0062] FIG. 1 and FIG. 2 are schematic diagrams of a real-time video protocol conversion system (10) according to an embodiment of the present disclosure.

[0063] Referring to FIGS. 1 and 2, a real-time video protocol conversion system (10) according to an embodiment of the present disclosure includes a conversion device (100), a first device (200), and a second device (300).

[0064] However, in some embodiments, the system (10) may include fewer or more components than those illustrated in FIG. 1.

[0065] The first device (200) is a camera device that captures images and refers to a transmitter.

[0066] The second device (300) is a device that processes images and refers to a receiving unit.

[0067] A real-time video protocol conversion device (100) automatically converts an asynchronous video communication protocol in real time by converting a first image of a first protocol received from a first device (200) into a second image of a second protocol and providing the converted image.

[0068] FIG. 3 is a block diagram of a real-time video protocol conversion device (100) according to an embodiment of the present disclosure.

[0069] Referring to FIG. 3, a real-time video protocol conversion device (100) according to an embodiment of the present disclosure includes a processor (110), a communication unit (120), and a memory (130).

[0070] However, in some embodiments, the video protocol conversion device (100) may include fewer or more components than the components illustrated in FIG. 2.

[0071] The processor (110) may be implemented as a storage unit that stores data regarding an algorithm for controlling the operation of components within the device or a program that reproduces the algorithm, and at least one processor (110) that performs the aforementioned operation using the data stored in the storage unit. In this case, the storage unit and the processor (110) may each be implemented as separate chips. Alternatively, the storage unit and the processor (110) may be implemented as a single chip.

[0072] In addition, the processor (110) can control any one or a combination of the components described above to implement various embodiments according to the present disclosure described in the drawings below on the device.

[0073] In addition to operations related to the above-described application, the processor (110) can typically control the overall operation of the device. The processor (110) can process signals, data, information, etc. input or output through the components described above, or run application programs stored in the storage unit, thereby providing or processing appropriate information or functions to the user.

[0074] In addition, the processor (110) may control at least some of the components of the device to run an application program stored in the storage unit. Furthermore, the processor (110) may operate at least two or more of the components included in the device in combination to run the application program.

[0075] The processor (110) may be implemented as one or more. Hereinafter, even if the processor (110) is expressed as singular, it may be considered as plural. The processor (110) may control the configurations of the video protocol conversion device (100). The processor (110) may refer to a data processing device built into hardware that has a physically structured circuit to perform a function expressed by a code or command included in a program. As such, the processor (110) is an example of a data processing device built into hardware, and may encompass processing devices such as a microprocessor (110), a central processing unit (CPU), a processor (110) core, a multiprocessor (110), an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto. The processor (110) may be equipped with a separate learning processor (110) for performing artificial intelligence operations, or may be equipped with a learning processor (110) on its own.

[0076] The communication unit (120) may include one or more modules that connect the video protocol conversion device (100) to one or more networks.

[0077] The communication unit (120) may include one or more components that enable communication with an external device, and may include, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0078] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a Digital Visual Interface (DVI), RS-232 (recommended standard 232), power line communication, or plain old telephone service (POTS).

[0079] The wireless communication module may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a WiFi module and a Wireless Broadband module.

[0080] The wireless communication module may include a wireless communication interface including an antenna and a transmitter for transmitting communication signals. Furthermore, the wireless communication module may further include a signal conversion module that modulates a digital control signal output from the processor (110) through the wireless communication interface into an analog wireless signal under the control of the processor (110).

[0081] The short-range communication module is for short-range communication, and can support short-range communication using at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0082] The storage unit can store data supporting various functions of the device. The storage unit can store a number of application programs (or applications) running on the device, data for the operation of the device, and commands. At least some of these application programs may exist for the basic functions of the device. Meanwhile, the application programs can be stored in the storage unit, installed on the device, and driven by the processor (110) to perform operations (or functions).

[0083] The storage unit can store data supporting various functions of the device, programs for the operation of the processor (110), input / output data (e.g., music files, still images, moving images, etc.), and a plurality of application programs (or applications) run on the device, data for the operation of the device, and commands. At least some of these application programs can be downloaded from an external server via wireless communication.

[0084] The storage unit may include at least one type of storage medium among a flash memory (130) type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (130) (e.g., an SD or XD storage unit, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory (130), a magnetic disk, and an optical disk. In addition, the storage unit may be a database that is separate from the device but is connected by wire or wirelessly.

[0085] The memory (130) may be electrically connected to the processor (110) and may store at least one code executed by the processor (110). The memory (130) may collectively refer to various types of storage devices. The memory (130) may store information necessary for performing operations using artificial intelligence, machine learning, and artificial neural networks.

[0086] The memory (130) can store various learning models. The learning models stored in the memory (130) can infer result values ​​for new input data other than learning data, and the inferred values ​​can be used as a basis for judgment to perform a certain action. The learning models stored in the memory (130) can perform learning based on label information, and various backpropagation algorithms can be applied so that the loss function has a target value to increase the accuracy of learning.

[0087] Additionally, the storage unit may have multiple processes for the video protocol conversion device (100).

[0088] In addition, the video protocol conversion device (100) may further include components such as an input unit, an output unit, and an interface unit.

[0089] The input unit is for inputting video information (or signal), audio information (or signal), data, or information input from a user, and may include at least one camera, at least one microphone, and at least one user input unit. Voice data or image data collected from the input unit may be analyzed and processed into a user control command.

[0090] The input unit is for receiving information from the user, and when information is input through the input unit, the processor (110) can control the operation of the device to correspond to the input information. The input unit may include a hardware physical key (e.g., a button located on at least one of the front, rear, and side of the device, a dome switch, a jog wheel, a jog switch, etc.) and a software touch key. As an example, the touch key may be a virtual key, a soft key, or a visual key displayed on a touch screen type display unit through software processing, or may be a touch key disposed on a part other than the touch screen. Meanwhile, the virtual key or visual key may have various forms and be displayed on the touch screen, and may be, for example, formed of a graphic, text, an icon, a video, or a combination thereof.

[0091] The output unit is for generating output related to visual, auditory, or tactile sensations, and may include at least one of a display unit, an audio output unit, a haptic module, and an optical output unit. The display unit may be formed as a layer structure with a touch sensor or formed as an integral part, thereby implementing a touch screen. Such a touch screen may function as a user input unit that provides an input interface between the device and a user, and at the same time, may provide an output interface between the device and the user.

[0092] The interface unit serves as a passageway for various types of external devices connected to the device. The interface unit may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module (SIM), an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. The device may perform appropriate control related to the external device connected to the interface unit.

[0093] Figure 4 is an example diagram for DMT Video Timing Parameter Definitions.

[0094] Figure 5 is an example diagram for Negative H & Positive V Syncs in DMT Video Timing Parameter Definitions.

[0095] Figure 6 is a drawing illustrating a DVP (Digital Video Port) Interface corresponding to the second device (300).

[0096] First, the timing parameters will be explained with reference to FIGS. 4 to 6.

[0097] Referring to FIGS. 4 and 5, each parameter can be defined as follows.

[0098] Valid video data begins to be output from the point where Vertical Active Start and Horizontal Active Start match.

[0099] Vertical Response Period (VSA, Vertical Sync Active): Sync period of the vertical synchronization signal (VSync), the period when VSync is active high.

[0100] 1st Vertical Back Porch Period (VBP): This is the back porch period of the vertical synchronization signal (VSync), and is the period from when the vertical synchronization signal (VSync) becomes Active low until Active Video is output.

[0101] Vertical Active Period (VACT): This is the active video period of VSync, and is the period where actual valid video data is output.

[0102] Second vertical front porch period (VFP): Front porch period of Vsync, from after VACT to before VSA

[0103] Horizontal Response Period (HSA, Horizontal Sync Active): Sync period of the horizontal synchronization signal (HSync), the period when HSync is active high.

[0104] 1st Horizontal Back Porch Period (HBP): This is the back porch period of HSync, which is the period from when HSync becomes Active low until Active Video is output.

[0105] Horizontal Active Period (HACT): This is the active video period of HSync, and is the period where actual valid video data is output.

[0106] Second Horizontal Front Porch Period (HFP): Front porch section of HSync, from after HACT to before HSA.

[0107] Referring to Fig. 6, T1 is VBP, T2 is HSA, T3 is HBP, T4 is T2+T3 (HSIZE: size of entire Horizontal section), T5 is VFP, and T6 is VSA.

[0108] FIG. 7 is a diagram illustrating an analysis of protocol differences between the first device (200) of FIG. 4 and the second device (300) of FIG. 7.

[0109] Referring to FIG. 7, in the embodiment of the present disclosure, the first device (200) corresponds to the VESA protocol, and the second device (300) corresponds to the DVP protocol.

[0110] A real-time video protocol conversion device (100) according to an embodiment of the present disclosure analyzes the difference between the first protocol of the first device (200) and the second protocol of the second device (300), and when a plurality of parameter values ​​for the first protocol are measured based on the difference, the device converts the first image into a second image of the second protocol based on the measured parameter values.

[0111] Referring to Figure 7, the differences between the VESA protocol and the DVP protocol are as follows.

[0112] For VESA, Vsync and Hsync occur simultaneously at the start of a frame. On the other hand, for DVP, Vsync occurs at the start of a frame, and Hsync occurs thereafter.

[0113] For VESA, data enable does not need to occur when hsync occurs. Data enable occurs after hsync occurs. On the other hand, for DVP, data enable occurs for the same length as hsync.

[0114] In the case of VESA, multiple data can be sent in parallel during the HACT period. For example, in the case of RGB, three pixel data (R, G, B) are sent simultaneously per pixel clock.

[0115] In this case, the data enable interval is reduced by approximately 1 / 3 compared to the HACT interval. In the case of DVP, only one pixel data can be received serially. The HACT length is equal to the data enable length.

[0116] For VSA, data enable may not occur continuously. In other words, data enable may not occur in the middle. For DVP, hsync and data enable are continuous.

[0117] FIG. 8 is a flowchart of a real-time video protocol conversion method according to an embodiment of the present disclosure.

[0118] FIGS. 9 to 16 are various exemplary drawings for explaining a real-time video protocol conversion device (100), method, and program according to an embodiment of the present disclosure.

[0119] With reference to FIGS. 8 to 16, a real-time video protocol conversion device (100), method, and program according to an embodiment of the present disclosure will be described.

[0120] The processor (110) detects the start of a new video line of the first protocol image received from the first device (200) through the communication unit (120). (S100)

[0121] The processor (110) receives a first image of a first protocol from a first device (200) that captures an image.

[0122] And, the processor (110) finally provides the image of the second protocol, into which the protocol has been converted, to the second device (300), which is an image processing device.

[0123] The real-time video protocol conversion device (100) according to the embodiment of the present disclosure has the advantage of being able to significantly reduce the memory (130) resources used by analyzing the video in line units rather than in frame units.

[0124] The processor (110) detects the edge of the vertical synchronization signal (Vsync) and the horizontal synchronization signal (Hsync) of the first protocol image and the data section (Data Enable). (S200)

[0125] Fig. 9 is a drawing illustrating an edge detector (140).

[0126] Referring to FIG. 9, a video protocol conversion device (100) according to an embodiment of the present disclosure includes at least one edge detector (140).

[0127] The processor (110) can detect the positive edge of the vertical synchronization signal (Vsync) and the negative edge of the horizontal synchronization signal (Hsync) of the first protocol image using the edge detector (140).

[0128] The processor (110) can measure values ​​for a plurality of first timing parameters based on the results detected through the edge detector (140) and the data interval (Data Enable).

[0129] The processor (110) measures values ​​for a plurality of first timing parameters based on the results detected by S200. (S300)

[0130] Referring to FIGS. 10 to 14, the video protocol conversion device (100) according to an embodiment of the present disclosure may further include a plurality of measurement modules.

[0131] Figure 10 is a drawing illustrating an HSA measurement module (151).

[0132] Fig. 11 is a drawing illustrating an HBP measurement module (152).

[0133] Fig. 12 is a drawing illustrating a PIXEL CNT measurement module (153).

[0134] Fig. 13 is a drawing illustrating an HSIZE measurement module (154).

[0135] Fig. 14 is a drawing illustrating an HFP measurement module (155).

[0136] The processor (110) can measure the horizontal response period (HAS, Horizontal Sync Active) using the HSA measurement module (151).

[0137] The processor (110) can measure the first horizontal back porch period using the HBP measurement module (152).

[0138] The processor (110) can count pixels using the PIXEL CNT measurement module (153).

[0139] The processor (110) can measure HSIZE (entire horizontal section) using the HSIZE measurement module (154).

[0140] That is, the processor (110) can measure the length of the Hsize (entire horizontal section), the horizontal response period (HSA, Horizontal Sync Active), the first horizontal porch period (HBP, Horizontal Back Porch), the horizontal active section (HACT, Horizontal Active), and the PIXEL_CNT section using the edge detector (140).

[0141] In addition, the processor (110) can calculate the HFP (Horizontal Front Porch) based on the measured result using the HFP measurement module (155).

[0142] For HFP, the remaining value is determined by subtracting HSA, HBP, and PIXEL_CNT from the entire horizontal section, HSIZE. This is because HFP is the section from the end of HACT to the start of the next Hsync, and it can be thought that detecting the negative edge of Data Enable is sufficient to determine the end of HACT. However, since pixel enables may not be continuous, it may be difficult to determine which data enable is the last. Therefore, since the actual number of valid data is the HACT length, the number of HFPs can be obtained by subtracting HSA, HBP, and PIXEL_CNT from the total.

[0143] That is, the processor (110) can calculate the HFP (Horizontal Front Porch) based on the following mathematical expression 1.

[0144]

[0145] Fig. 15 is a block diagram illustrating a part of the configuration of a video protocol conversion device (100).

[0146] Referring to FIG. 15, in one embodiment, the processor (110) can store input Pixel data in memory through an Asynchronous FIFO.

[0147] And, based on the first timing parameter analyzed for the first protocol of the first device (200), the second timing parameter for the second protocol of the second device (300) can be set.

[0148] The processor (110) sets a second timing parameter for image conversion based on the value of the first timing parameter measured by S300. (S400)

[0149] The processor (110) converts the first protocol image into a second protocol image based on the second timing parameter set in S400. (S500)

[0150] In an embodiment of the present disclosure, the storage unit may store differences between the first protocol and the second protocol.

[0151] Using an edge detector (140), a positive edge of a vertical synchronization signal (Vsync) and a negative edge of a horizontal synchronization signal (Hsync) of a first protocol image can be detected, and values ​​for a plurality of first timing parameters can be measured based on the results detected through the edge detector (140) and a data section (Data Enable).

[0152] In the first protocol video, when the frame starts, the vertical sync signal and the horizontal sync signal occur simultaneously, and in the second protocol video, when the frame starts, the vertical sync signal occurs and then the horizontal sync signal occurs.

[0153] In the first protocol image, a data interval occurs after a horizontal synchronization signal occurs, and in the second protocol image, a data interval of the same length as the horizontal synchronization signal occurs.

[0154] Figure 16 is a drawing illustrating the operation process of a video protocol conversion device (100).

[0155] Referring to Fig. 16, the operation process of the video protocol conversion device (100) will be described.

[0156] ① When a parameter update signal for a new line is input in the IDLE state, the video protocol conversion device (100) transitions to the VSA state and Vsync of the DVP occurs. At this time, the length of the VSA section is maintained high until there is valid data in the new line.

[0157] ② The video protocol conversion device (100) drops the Vsync of the DVP to low while transitioning to the VBP state when valid data exists on a new line. At this time, the length of the VBP section is determined by the sum of the HSA and the HBP.

[0158] ③ The video protocol conversion device (100) sends Hsync and data_valid to high when the VBP transitions to the HACT state at the end of the section, and reads data previously stored in the FIFO and sends it together.

[0159] ④ After the video protocol conversion device (100) sends all valid data of a line up to the length of HACT, if valid data exists in a new line, it transitions to the HBLANK state, and if not, it transitions to the IDLE state. The length of the HBLANK section is determined by the HFP size, and since the operating speeds of the two are different, the size of the HBLANK may increase compared to the HFP size until a parameter update signal for a new line is input. When parallel data is input, the data enable length is increased by the number of input parallel data, and it is serialized and sent.

[0160] In one embodiment, the processor (110) can solve an asynchrony problem caused by a difference in Clock Speed ​​between the first device (200) and the second device (300) by using an Asynchronous FIFO and Synchronizer.

[0161] The processor (110) can synchronize the control signal to the clock of the DVP by using a synchronizer to synchronize the signal to the clock of the VESA.

[0162] When a plurality of image data are output from the first device (200), the processor (110) can convert the plurality of image data into serial single image data using a serializer and transmit the converted image data to the second device (300).

[0163] Through such configurations, the real-time video protocol conversion device (100) according to the embodiment of the present disclosure can process images in the receiving unit by converting the video communication protocol of the transmitting unit to suit the receiving unit, and has the advantage of being able to flexibly adjust performance because the transmitting unit and the receiving unit can independently change the synchronization speed.

[0164] In addition, the real-time video protocol conversion device (100) according to the embodiment of the present disclosure has the advantage of using a significantly smaller amount of memory (130) because it analyzes one line rather than one frame.

[0165] In addition, the real-time video protocol conversion device (100) according to the embodiment of the present disclosure automatically converts the protocol in real time because the video timing parameters are set based on a signal directly received without the CPU setting the parameters.

[0166] Additionally, the real-time video protocol conversion device (100) according to an embodiment of the present disclosure may provide real-time automatic video protocol conversion for multiple input cameras.

[0167] The method according to one embodiment of the present disclosure described above can be implemented as a program (or application) and stored in a medium to be executed in combination with a hardware server.

[0168] The above-described program may include codes coded in a computer language, such as C, C++, JAVA, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as a program. Such codes may include functional codes related to functions that define functions necessary for executing the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to send and receive during communication.

[0169] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0170] The steps of a method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0171] While the embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will appreciate that the present disclosure can be implemented in other specific forms without altering the technical spirit or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. Department of Communications; A storage unit storing at least one instruction; and Includes a processor, The above processor executes at least one instruction When the start of a new video line of the first protocol image received through the above communication unit is detected, Detecting the edge and data section (Data Enable) of the vertical synchronization signal (Vsync) and horizontal synchronization signal (Hsync) of the first protocol image, Based on the above detected results, values ​​for multiple first timing parameters are measured, Set a second timing parameter for image conversion based on the value of the first timing parameter measured above, Characterized in that the first protocol image is converted into a second protocol image based on the second timing parameter set above. Real-time video protocol conversion device.

2. In paragraph 1, The above processor, Receiving the first protocol image from the first device that captures the image, The second protocol image is provided to the second device, which is an image processing device. It is characterized by solving the asynchronous problem caused by the difference in Clock Speed ​​between the first device and the second device by using Asynchronous FIFO and Synchronizer. Real-time video protocol conversion device.

3. In paragraph 2, The above processor, When a plurality of image data are output from the first device, the plurality of image data are converted into serial single image data and transmitted to the second device. Real-time video protocol conversion device.

4. In paragraph 1, The above storage unit stores the differences between the first protocol and the second protocol, The above processor, Using an edge detector, the positive edge of the vertical synchronization signal (Vsync) and the negative edge of the horizontal synchronization signal (Hsync) of the first protocol image are detected, Characterized in that the values ​​for the plurality of first timing parameters are measured based on the results detected through the edge detector and the data interval (Data Enable). Real-time video protocol conversion device.

5. In paragraph 4, In the above first protocol video, when the frame starts, the vertical synchronization signal and the horizontal synchronization signal occur simultaneously, The above second protocol image is characterized in that when the Frame starts, a vertical synchronization signal occurs followed by a horizontal synchronization signal. Real-time video protocol conversion device.

6. In paragraph 4, The above first protocol image has a data section that occurs after a horizontal synchronization signal occurs, The above second protocol image is characterized in that a data section of the same length as the horizontal synchronization signal occurs. Real-time video protocol conversion device.

7. In paragraph 4, The above processor, Using the above edge detector, the length of Hsize (entire horizontal section), horizontal response period (HSA, Horizontal Sync Active), first horizontal porch period (HBP, Horizontal Back Porch), horizontal active section (HACT, Horizontal Active), and PIXEL_CNT section are measured. Characterized in that the HFP (Horizontal Front Porch) is calculated based on the above measured results. Real-time video protocol conversion device.

8. In paragraph 7, The above processor, It is characterized by calculating HFP (Horizontal Front Porch) based on the following mathematical formula 1. Real-time video protocol conversion device. [Mathematical Formula 1] 9. A method performed by a video protocol conversion device, A step of detecting the start of a new video line of a first protocol image received through a communication unit; A step of detecting an edge and a data section (Data Enable) of a vertical synchronization signal (Vsync) and a horizontal synchronization signal (Hsync) of the first protocol image; A step of measuring values ​​for a plurality of first timing parameters based on the above detected results; A step of setting a second timing parameter for image conversion based on the value of the first timing parameter measured above; and A step of converting the first protocol image into a second protocol image based on the second timing parameter set above, Method for converting real-time video protocols.

10. A computer-readable recording medium having a program stored thereon, which is combined with a computer as hardware and executes the method of claim 9.

Images