Real-time video frame insertion device, system and method

Abstract

The invention discloses a real-time video frame insertion device, system and method, and the device comprises a data receiving and storing module which is used for receiving each original video frame and a control signal of an original video, and storing the original video frames; the time sequence generation module is used for generating a Stream time sequence of n video frames after frame insertion corresponding to the original video frame according to the resolution of the original video, and sending a data reading control signal to the data reading module according to the time sequence; the data reading module is used for successively reading n video frames from the storage address of the original video frame according to the control signal of the original video frame, the data reading control signal and the resolution; the control signal of the original video frame represents a frame start address of each video frame when n video frames are read out; and the time sequence generation module is used for sending the n video frames and the time sequence to the video output module for outputting. According to the method, the controllability and real-time performance of video frame insertion can be improved, and the method has good expansibility and low cost.

Description

Technical Field

[0001] The present invention belongs to the technical field of video frame interpolation, and particularly relates to a real-time video frame interpolation device, system and method. Background Technique

[0002] Video frame interpolation is an important part of infrared imaging real-time simulation technology, which is mainly used for scene simulation and imaging device simulation in the fields of industrial inspection, scientific research, equipment, etc. Its core purpose is to simulate and generate higher-quality intermediate frames according to adjacent frames of the original video through algorithms and calculations to meet the application requirements in different scenarios. The process usually includes the following key links: video acquisition, preprocessing, algorithm design (motion estimation, interpolation method, etc.), and image output, etc.

[0003] Real-time simulation: Emphasize the real-time nature of simulation processing and is applied to scenarios such as on-line testing of equipment and algorithm verification. This type of simulation needs to complete complex calculations in a short time, provide a response speed close to the actual environment for the device under test, and pose higher requirements for the efficiency of the algorithm and the performance of the hardware.

[0004] Currently, most existing solutions for infrared imaging real-time simulation video frame interpolation are based on optical flow method, deep learning, etc. Optical flow is based on flow optical flow (the instantaneous velocity of the pixels of a moving object in space on the observation imaging plane), and a small part based on network structures such as kernel estimation, deformable convolution, and attention also achieves relatively good performance and inference speed. The optical flow method realizes video frame interpolation by calculating the displacement information of pixel points in the image, which can improve the smoothness and visual effect of the video to a certain extent. The role of deep learning methods in video frame interpolation: By using deep learning models such as neural networks, the spatio-temporal features in the video can be learned, and realistic interpolation frames can be generated. In the video frame interpolation task, usually paired consecutive video frames are used as input, and the true value of the intermediate frame is used as the target for training. Through data-driven learning and a powerful network structure, deep learning methods can effectively learn the relationship and motion pattern between video frames, thereby achieving high-quality video frame interpolation.

[0005] Optical flow methods involve preprocessing, optical flow estimation or training with a training set, filtering, convolution, and subsequent processing, etc., which are overall quite complex. In addition, the optical flow field does not necessarily reflect the actual motion of the target. Therefore, optical flow methods are sensitive to light, and light changes can easily affect the recognition effect. Dense optical flow requires calculating the offset of each pixel point, resulting in a large amount of calculation and poor timeliness. Optical flow methods are computationally complex and time-consuming, and it is difficult to achieve real-time detection unless there is special hardware support. Deep learning models usually require a large amount of labeled data for training to learn the complex relationships between video frames. Training deep learning models requires a large amount of computing resources, including high-performance graphics processing units (GPUs). Deep learning models perform well on training data, but may experience a decline in performance when faced with new, unseen video data. This is because the model may overfit the training data and lack generalization ability.

[0006] That is to say, the existing video frame interpolation schemes currently have high computational complexity, low real-time performance, low scalability, low controllability, and high cost. Summary of the Invention

[0007] To solve the above problems existing in the prior art, the present invention provides a real-time video frame interpolation device, system, and method.

[0008] The technical problems to be solved by the present invention are achieved through the following technical solutions:

[0009] The present invention provides a real-time video frame interpolation device, including:

[0010] A data receiving and storing module, configured to receive each original video frame and the control signal of each original video frame in the original video that needs frame interpolation sent by the master control device, and store each received original video frame;

[0011] A timing generation module, configured to generate a Stream timing of n interpolated video frames corresponding to the currently received original video frame according to the resolution of the original video after the data receiving and storing module stores the currently received original video frame, and send a data reading control signal to the data reading module according to the Stream timing; n is a preset positive integer greater than 1;

[0012] The data reading module is configured to sequentially read out n video frames from the storage address of the currently received original video frame according to the control signal of the currently received original video frame, the data reading control signal, and the resolution, and send them to the timing generation module; wherein, the control signal of the currently received original video frame is used to represent the frame start address of each video frame when reading out the n video frames, and the frame start addresses of the n video frames are different;

[0013] The timing generation module is further configured to send the n video frames and the Stream timing of the n video frames to the video output module for output.

[0014] The present invention also provides a real-time video frame interpolation method, including:

[0015] S1. Receiving a current original video frame in an original video that needs frame interpolation and a control signal of the current original video frame sent by a master control device;

[0016] S2. Storing the current original video frame, and after the storage of the current original video frame is completed, generating the Stream timing of n interpolated video frames corresponding to the current original video frame according to the resolution of the original video, and generating a data read control signal according to the Stream timing; n is a preset positive integer greater than 1;

[0017] S3. Sequentially reading out n video frames from the storage address of the current original video frame according to the control signal of the current original video frame, the data read control signal, and the resolution, encoding each read video frame, outputting the encoded n video frames according to the Stream timing of the n video frames, and after reading out the nth video frame, receiving the next original video frame in the original video and the control signal of the next original video frame sent by the master control device, and then returning to the above S2 to continue execution until the last original video frame in the original video is processed; wherein, the control signal of the current original video frame is used to represent the frame start address of each video frame when reading out the n video frames, and the frame start addresses of the n video frames are different; the video composed of all the output video frames is the interpolated video of the original video.

[0018] The present invention also provides a real-time video frame interpolation system, including: a real-time video frame interpolation device, a master control device, and a position information generation device;

[0019] The master control device is configured to sequentially send an original video frame in an original video that needs frame interpolation to the real-time video frame interpolation device, wherein one original video frame is sent each time;

[0020] The real-time video frame interpolation device is configured to store each received original video frame and send it to the position information generation device;

[0021] The position information generation device is configured to generate the position information of a target in the original video frame according to each received original video frame, and send the position information of the target in the original video frame to the master control device;

[0022] The master control device is further configured to generate a control signal of the original video according to the position information of the target in the original video frame and send the control signal to the real-time video frame interpolation device;

[0023] The real-time video frame interpolation device is further configured to, after storing each received original video frame, generate a Stream timing of n interpolated video frames corresponding to the originally received video frame according to the resolution of the original video, generate a data reading control signal according to the Stream timing, and successively read out n video frames from the storage address of the originally received video frame according to the control signal of the originally received video frame, the data reading control signal, and the resolution, and after converting the format of each read video frame, output the n video frames according to the Stream timing of the n video frames; n is a preset positive integer greater than 1.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] The present invention realizes video frame interpolation by using a pure hardware solution, which can be independent of a computer system or an image acquisition system, and has the characteristics of low cost, low latency, and easy modular addition and subtraction. Moreover, without interacting with a computer or an image acquisition system, only by relying on the control signal sent by the master control device, the improvement of the image video frame rate can be realized. Since the frame start address can be controlled by external input, the interpolated image and its frame rate can be generated according to requirements without additional complex algorithms, and it also has the characteristic of easy expansion. The real-time video frame interpolation device can be implemented based on an FPGA. By virtue of the reconfigurable and parallel characteristics of the FPGA, the hardware can repeatedly update the loaded algorithm, thereby reducing the iteration cost and having strong scalability. That is to say, aiming at the requirement of video frame interpolation in real-time infrared imaging simulation, the present invention proposes an innovative solution, which significantly improves the controllability and real-time performance of video frame interpolation, and has good scalability and low cost, providing valuable technical support for applications in related fields.

[0026] The following will further elaborate on the present invention in detail with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of the technical principle of the traditional frame interpolation method;

[0028] Figure 2 is a schematic diagram of the frame interpolation principle of the real-time video frame interpolation device of the present invention;

[0029] Figure 3 is a schematic structural diagram of the real-time video frame interpolation device provided by the present invention;

[0030] Figure 4 It is a schematic structural diagram of a real-time video frame interpolation system provided by the present invention. Specific Embodiments

[0031] The following further describes the present invention in detail with reference to specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0032] An embodiment of the present invention provides a real-time video frame interpolation device, which includes: a data reception and storage module, a timing generation module, a data reading module, and a video output module.

[0033] The data reception and storage module is used to receive each original video frame and the control signal of each original video frame in the original video that needs frame interpolation sent by the master control device, and store each received original video frame. The timing generation module is used to generate the Stream timing of n interpolated video frames corresponding to the currently received original video frame according to the resolution of the original video after the data reception and storage module stores the currently received original video frame, and send a data reading control signal to the data reading module according to the Stream timing; n is a preset positive integer greater than 1. The data reading module is used to sequentially read out n video frames from the storage address of the currently received original video frame according to the control signal, data reading control signal, and resolution of the currently received original video frame, and send them to the timing generation module; wherein, the control signal of the currently received original video frame is used to represent the frame start address of each video frame when reading out n video frames, and the frame start addresses of the n video frames are different. The timing generation module is further used to send the n video frames and the Stream timing of the n video frames to the video output module for output.

[0034] Here, the frame rate achieved by the interpolated video is the product of the frame rate of the original video and n, and the value of n can be set according to actual needs. The n interpolated video frames corresponding to the currently received original video frame indicate that the currently received original video frame needs to be interpolated into n video frames. The frame start address of each video frame represents the position of the pixel from which to start reading in the currently received original video frame when reading out the video frame, that is, the frame start address of each video frame represents from which pixel in the currently received original video frame to start reading pixel data. When the frame start addresses are different, the position of the target in a read video frame in the video frame is different. After reading one frame of data and then continuing to read the second frame of data, the frame start address is changed, so that the position of the target relative to the first frame in the background will change. In this way, by using n different frame start addresses and according to the currently received original video frame, n video frames with different target positions can be generated, and the effect of frame rate improvement can be achieved.

[0035] The following throughFigure 1 and Figure 2 The technical principle of the present invention and the technical principle of the traditional frame interpolation method will be described respectively. When the video frame rate of the original video is 60HZ and needs to reach 240HZ, the frame rate needs to be increased by 3 times, that is, 3 image frames need to be newly inserted. When 3 image frames need to be inserted between the first and the second original video frames of the original video, and the starting positions of the targets in these 3 image frames to be inserted are point A, point B, and point C respectively, as Figure 1 shown, the traditional frame interpolation method will read data according to the fixed read frame start address G, and use some algorithms to generate these 3 intermediate frames to be inserted based on the read data. And as Figure 2 shown, the present invention can directly read out these 3 intermediate frames to be inserted by using 3 different read frame start addresses G 0 、G 1 、G 2 .

[0036] In the present invention, the timing generation module starts the generation of the read timing after waiting for a frame of data to be completely written to the specified position in the DDR, avoiding the problem of read-write conflict.

[0037] In the present invention, the control signal of each original video frame is generated by the master control device according to the position information of the target in the original video frame. Correspondingly, the data receiving and storing module is further configured to send each received original video frame to the position information generating device, so that the position information generating device generates the position information of the target in each received original video frame according to each received original video frame.

[0038] In some embodiments, the device further includes a DDR storage module. The data reception and storage module includes: a data conversion module, a parsing module, a write VDMA module, an interconnection module, and a MIG. The data conversion module is configured to receive the differential data stream sent by the master control device, convert the differential data stream into data in Stream format, and send the data in Stream format to the parsing module. The parsing module is configured to parse the data in Stream format, and when the parsed data in Stream format is an original video frame in the original video, send the parsed original video frame to the write VDMA module; when the parsed data in Stream format is a control signal of an original video frame, send the control signal to the data reading module. The write VDMA module is configured to send each received original video frame to the interconnection module; the interconnection module is configured to convert the data format of each received original video frame into the data format required by the MIG and then send it to the MIG. The MIG is configured to store each received original video frame in the DDR storage module. It should be noted that the data input to the parsing module includes image data and control data (the control signal of the above-mentioned original video frame), and both types of data packets include an identifier and content data. The parsing module will determine whether the data is image data or control data according to the identifier. If it is image data, it will be output to the write VDMA module and finally written into the DDR. If it is a control signal, it will act on the read VDMA module to control the read start address of the read VDMD module.

[0039] The write VDMA module is used to efficiently write data into an external DDR storage module, supports the burst transfer mode, and can quickly and continuously transfer a large amount of data. By means of burst writing, the efficiency of data writing can be improved, the access time to the DDR storage module can be reduced, and thus the performance of the entire device can be improved. The MIG realizes the function of data interaction with the memory storage chip and is used for reading and writing with an external DDR storage module. It receives external read / write start signals, addresses, and data length information, and performs the AXI bus interaction process. It can be parameterized configured according to the characteristics of specific memory chips, including clock frequency, data width, address mapping, etc., to achieve the best performance and compatibility.

[0040] Exemplarily, the modules other than the DDR storage module in the real-time video frame interpolation device provided by the present invention can be implemented by an FPGA. Figure 3 is a schematic structural diagram of the real-time video frame interpolation device provided by the present invention, as Figure 3As shown, the data conversion module is the Aurora 64 / 66 IP core, the DDR storage module is the DDR chip, and the video output module is also the Aurora 64 / 66 IP core. The Aurora 64 / 66 IP core is an IP core of a high-speed serial communication interface protocol used for data exchange between electronic devices such as FPGA chips. It mainly has the following functions: 1. Data format conversion: It converts differential data and parallel data into each other. Here, the parallel data specifically refers to Stream stream data. For reception, the Aurora 64 / 66 IP core converts differential signals into Stream stream data, and vice versa for transmission; 2. Provide a flexible data interface: It supports full-duplex and simplex channels, provides user interfaces for AXI4 stream and Stream stream reception (RX) and transmission (TX), and cooperates with different types of data interface standards and protocols to communicate with external devices of various different architectures or protocols, realizing data interaction between different systems and chips, and enhancing the versatility and compatibility of products.

[0041] In some embodiments, the write VDMA module is also used to send a completion signal to the timing generation module after the MIG stores an original video frame in the DDR storage module, so that the timing generation module generates Stream timing and data read control signals.

[0042] In some embodiments, as Figure 3 As shown, the data readout module includes: an interconnection module, MIG, and a read VDMA module. The read VDMA module is used to obtain the frame start addresses of n video frames to be read out according to the control signals of the original video frame received this time after receiving the data read control signal, and each time it sends a frame start address to the interconnection module, and then sends the next frame start address to the interconnection module through a VDMA interrupt. The interconnection module is also used to send each received frame start address to the MIG. The MIG is also used to read out a video frame from the DDR storage module corresponding to the resolution and each received frame start address and return it to the interconnection module. The interconnection module is also used to perform data format conversion on each video frame returned by the MIG and then send it to the read VDMA module. The read VDMA module is also used to return each received video frame to the timing generation module.

[0043] In some embodiments, the read VDMA module is also used to send a write control signal to the write VDMA module after receiving the nth video frame to control the write VDMA module to store the next original video frame of the original video received this time in the DDR storage module through the interconnection module and the MIG.

[0044] In some embodiments, in a DDR storage module, the storage addresses of different original video frames are different. Based on this, the storage read and write process of the present invention is as follows: Write the image data of the first frame of the original video into the first frame buffer interval in the DDR. To avoid image overlay and reading from an address where data has not been written, after the image data of the first frame is completely written into the first frame buffer interval, start generating a read timing to read the data in the first frame buffer interval. After one frame is read, change the read start address (i.e., the above-mentioned frame start address), and continue to read the data in the first frame buffer interval according to the changed read start address until the corresponding multiple frames are read from the first frame buffer interval according to the required read start address. Then, write the image data of the second frame of the original video into the second frame buffer interval in the DDR. After the image data of the second frame is completely written into the second frame buffer interval, start generating a read timing to read the data in the second frame buffer interval. After one frame is read, change the read start address, and continue to read the data in the second frame buffer interval according to the changed read start address until the corresponding multiple frames are read from the second frame buffer interval according to the required read start address. Then, write the image data of the third frame of the original video into the third frame buffer interval in the DDR, and repeat the above principle until the image data of the last frame of the original video is completed. Since both the read start address and the read / write length are parameterized, it has the characteristic of being easy to adapt to various resolutions.

[0045] The function of the timing generation module is to control the reading and output of video data and interact with other video processing modules. By generating a read request signal, image data can be read from an external data source, processed, and output. At the same time, by delaying the synchronization signal and data valid signal, the correct output of the image data and timing control can be ensured. By inputting different clock frequencies, timings with different frame rates can be generated for reading data with different frame rates. In the present invention, the timing generation module includes a FIFO for storing data, and the overall control of the timing generation module is controlled by a state machine. First, it is in the default state. When the write VDMA starts to write a frame of data into the DDR, a frame start signal is pulled high. At this time, the state machine jumps to the read waiting state. In this state, a counter waits for a period of time. After a frame of data is completely written into the DDR, it jumps to the read state. At this time, the output tready signal is pulled high, and this signal is output to the read VDMA to indicate a data reading request. The read VDMA will write the data into the timing generation module and store it in the FIFO. After a certain amount of data is in the FIFO, the data starts to be read out from the FIFO. After writing a line of data, the state machine jumps to the line end state. In this state, an interval count is performed, that is, there is a certain gap between rows. At the same time, a line count is performed for each read line. When the line count value is greater than or equal to the number of image columns, the state machine jumps to the next frame reading state, the frame interpolation count is incremented, and the line count value is reset to zero. If the line count value is less than the number of image columns, it jumps to the read state to continue reading the next line of data. When the state machine is in the next frame reading state, a frame interval count is performed, that is, there is a certain gap between rows. If the frame interpolation count is equal to the number of frames to be interpolated per frame, it jumps to the default state, indicating that the interpolation of a written frame is completed, and waits for the arrival of the frame start signal for the next frame written by the write VDMA into the DDR. If the frame interpolation count is less than the number of frames to be interpolated per frame and the frame interval count is completed, it jumps to the read waiting state.

[0046] The present invention also provides a real-time video frame interpolation method. This method is executed by the above device, and this method includes:

[0047] S1. Receive the current original video frame and the control signal of the current original video frame in the original video that needs to be frame interpolated sent by the master control device.

[0048] S2. Store the current original video frame, and after the storage of the current original video frame is completed, generate the Stream timing of n interpolated video frames corresponding to the current original video frame according to the resolution of the original video, and generate a data reading control signal according to the Stream timing; n is a preset positive integer greater than 1.

[0049] S3. According to the control signal, data reading control signal, and resolution of the current original video frame, successively read out n video frames from the storage address of the current original video frame, encode each read video frame, output the n encoded video frames according to the Stream timing of the n video frames, and after reading the nth video frame, receive the next original video frame of the current original video frame and the control signal of the next original video frame sent by the master control device, and then return to the above S2 to continue execution until the last original video frame in the original video is processed; wherein, the control signal of the current original video frame is used to represent the frame start address of each video frame when reading out n video frames, and the frame start addresses of the n video frames are different; the video composed of all the output video frames is the video after frame interpolation of the original video.

[0050] Since the above real-time video frame interpolation device has been described in detail, therefore, the execution principle of each step of the method will not be specifically described here.

[0051] The present invention also provides a real-time video frame interpolation system, as Figure 4 shown, including: the above real-time video frame interpolation device, a master control device, and a position information generation device. The master control device is used to send the original video frames in the original video that need to be frame interpolated to the real-time video frame interpolation device frame by frame, where one original video frame is sent each time. The real-time video frame interpolation device is used to store each received original video frame and send it to the position information generation device. The position information generation device is used to generate the position information of the target in the original video frame in the original video frame according to each received original video frame, and send the position information of the target in the original video frame in the original video frame to the master control device. The master control device is also used to generate the control signal of the original video according to the position information of the target in the original video frame in the original video frame and send it to the real-time video frame interpolation device. The real-time video frame interpolation device is also used to, after storing each received original video frame, generate the Stream timing of the n video frames after frame interpolation corresponding to the currently received original video frame according to the resolution of the original video, generate a data reading control signal according to the Stream timing, and according to the control signal, data reading control signal, and resolution of the currently received original video frame, successively read out n video frames from the storage address of the currently received original video frame, and after converting the format of each read video frame, output the n video frames according to the Stream timing of the n video frames; n is a preset positive integer greater than 1.

[0052] Exemplarily, the master control device can be an infrared real-time imaging simulation platform. Therefore, the present invention can solve the problem of increasing the frame rate of the output video in infrared imaging real-time simulation. Video frame interpolation in infrared imaging real-time simulation is a key part of infrared imaging real-time simulation technology. Video frame interpolation in infrared imaging real-time simulation generates intermediate frames in the original infrared video sequence through steps such as motion estimation, interpolation calculation, and fusion optimization, thereby increasing the frame rate and smoothness of the video while maintaining the characteristics and authenticity of infrared imaging, and then being used for display or subsequent algorithm processing. Infrared imaging systems usually have index requirements for the frame rate of images. When the simulation system is running, it is required to give brightness calculation data in real time. In such application cases, due to the performance problems of some software or hardware itself, it is technically difficult to rely on professional model software to give calculation results in real time, resulting in too low frame rate of the simulation video. The present invention can provide a processing system with both real-time performance and controllability for the video frame rate requirements of real-time infrared imaging simulation.

[0053] Specifically, after the master control device receives the position information from the position information generating device, it calculates the pixel position offset in combination with the moving speed of the target, that is, the control signal sent to the real-time frame interpolation device. This control signal contains two elements, direction and distance. The direction includes up, down, left, and right, and the distance represents the number of pixels.

[0054] Specifically, after the real-time video frame interpolation device sends the image frames to the position information generating device, the position information generating device generates position information based on these image frames using a detection and tracking algorithm. The detection and tracking algorithm has the functions of detection and prediction judgment. When the input image target is not in the middle of the image, such as when the target is on the left side of the image, the position information generated by the position information generating device for the master control device will be offset to the left to achieve the effect that the target always remains in the middle of the image.

[0055] It should be noted that the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of the present invention, the meaning of "a plurality" is two or more unless otherwise specifically defined.

[0056] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0057] In the specification, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude a plurality. Certain measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0058] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A real-time video frame insertion device, characterized in that: include: The data receiving and storing module is used to receive each original video frame in the original video that needs to be inserted and the control signal of each original video frame sent by the master control device, and store each received original video frame; A timing generation module is used to generate a stream timing of n video frames after the original video frame received this time is inserted and corresponding to the original video frame received this time according to the resolution of the original video after the data receiving and storing module stores the original video frame received this time, and send a data reading control signal to the data reading module according to the stream timing; n is a preset positive integer greater than 1; The data reading module is used to read out n video frames from the storage address of the original video frame received this time and send them to the timing generation module according to the control signal of the original video frame received this time, the data reading control signal and the resolution; wherein the control signal of the original video frame received this time is used to represent the frame start address of each video frame when reading out the n video frames, and the frame start addresses of the n video frames are different; The timing generation module is also used to send the n video frames and the Stream timing of the n video frames to the video output module for output.

2. The real-time video frame insertion device according to claim 1, characterized in that: The control signal of each original video frame is generated by the master control device according to the position information of the target in the original video frame; The data receiving and storing module is also used to send each received original video frame to the position information generating device, so that the position information generating device generates the position information of the target in each received original video frame according to each received original video frame.

3. The real-time video frame insertion device according to claim 1, characterized in that: The device also includes a DDR storage module, and the data receiving and storing module includes: A data conversion module, used for receiving the differential data stream sent by the master control device, converting the differential data stream into data in Stream format, and sending the data in Stream format to the parsing module; The parsing module is used to parse the data in Stream format, and when the data in Stream format is parsed to be an original video frame in the original video, send the parsed original video frame to the write VDMA module, and when the data in Stream format is parsed to be a control signal of an original video frame, send the control signal to the data reading module; The write VDMA module is used to send an original video frame received each time to the interconnection module; The interconnection module is used to convert the data format of an original video frame received each time into the data format required by the MIG and then send it to the MIG; The MIG is used to store an original video frame received each time into the DDR storage module.

4. The real-time video frame insertion device according to claim 3, characterized in that: The write VDMA module is also used to send a completion signal to the timing generation module after the MIG stores an original video frame in the DDR storage module, so that the timing generation module generates the Stream timing and the data reading control signal.

5. The real-time video frame insertion device according to claim 3, characterized in that: The data reading module includes: the interconnection module, the MIG and a VDMA reading module; The read VDMA module is used to obtain the frame start addresses of n video frames to be read out according to the control signal of the original video frame received this time after receiving the data read control signal, and send the next frame start address to the interconnection module through VDMA interrupt after sending a frame start address to the interconnection module each time; The interconnection module is further used to send the start address of each received frame to the MIG; The MIG is further used to read a video frame from the DDR storage module according to the resolution and the frame start address received each time, and return it to the interconnection module; The interconnection module is also used to convert the data format of the video frame returned by the MIG each time and send it to the read VDMA module; The VDMA reading module is also used to return a received video frame to the timing generation module each time.

6. The real-time video frame insertion device according to claim 5, characterized in that: The read VDMA module is also used to send a write control signal to the write VDMA module after receiving the nth video frame, so as to control the write VDMA module to store the next original video frame of the original video received this time in the DDR storage module through the interconnection module and the MIG.

7. The real-time video frame insertion device according to claim 1, characterized in that: The frame rate of the video after interpolation is the product of the frame rate of the original video and n.

8. The real-time video frame insertion device according to claim 3, characterized in that: In the DDR storage module, different original video frames have different storage addresses.

9. A real-time video frame insertion method, characterized in that: include: S1, receiving a current original video frame in an original video that needs frame insertion and a control signal of the current original video frame sent by a master control device; S2, storing the current original video frame, and after the current original video frame is stored, generating a stream timing of n video frames after the interpolation corresponding to the current original video frame according to the resolution of the original video, and generating a data reading control signal according to the stream timing; n is a preset positive integer greater than 1; S3. According to the control signal of the current original video frame, the data reading control signal and the resolution, n video frames are read out one by one from the storage address of the current original video frame, and each read video frame is encoded, and the encoded n video frames are output according to the Stream timing of the n video frames, and after reading out the nth video frame, the next original video frame of the current original video frame and the control signal of the next original video frame are received from the master control device, and then the execution is returned to the above S2 to continue until the last original video frame in the original video is processed; wherein, the control signal of the current original video frame is used to represent the frame start address of each video frame when reading out the n video frames, and the frame start addresses of the n video frames are different; the video composed of all the output video frames is the video after the interpolation of the original video.

10. A real-time video frame insertion system, characterized in that: include: Real-time video frame insertion device, master control equipment and position information generation device; The master control device is used to send original video frames in the original video that need to be interpolated to the real-time video interpolation device frame by frame, wherein one original video frame is sent each time; The real-time video frame insertion device is used to store and send an original video frame received each time to the position information generating device; The position information generating device is used to generate the position information of the target in the original video frame in the original video frame according to each received original video frame, and send the position information of the target in the original video frame in the original video frame to the master control device; The master control device is further used to generate a control signal of the original video according to the position information of the target in the original video frame and send the control signal to the real-time video frame insertion device; The real-time video interpolation device is also used to generate a Stream timing of n video frames after interpolation corresponding to the original video frame received this time according to the resolution of the original video after storing each received original video frame, generate a data reading control signal according to the Stream timing, read out n video frames one by one from the storage address of the original video frame received this time according to the control signal of the original video frame received this time, the data reading control signal and the resolution, and after format conversion of each read video frame, output the n video frames according to the Stream timing of the n video frames; n is a preset positive integer greater than 1.

Images