Video signal converter

Abstract

To obtain an image of a low frame rate with smooth motions when converting an image signal of a high frame rate to generate an image signal of a low frame rate.SOLUTION: One of representative image signal conversion devices comprises an image conversion unit which converts a first image signal of a high frame rate to a second image signal of a low frame rate. The image conversion unit adds a plurality of consecutive first frames of the first image signal to generate second frames included in the second image signal.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , , , , , ,

[0001] The present invention relates to a video signal conversion device.

Background Art

[0002] Conventionally, when switching and operating videos with a frame rate of 60p and videos with a frame rate of 24p in the same system, the camera side always operates with progressive scanning at a frame rate of 60p, while the control device side performs frame rate conversion from a frame rate of 60p to a frame rate of 24p. Patent Document 1 discloses the following system as a video signal conversion system that converts a 4K video signal with a frame rate of 60p captured by a television camera into a video signal with a frame rate of 24p using a frame memory. "A camera control device having a serial-parallel conversion unit, a buffer memory unit, a 24P converter unit, a pixel interpolation unit, and a parallel-serial conversion unit, wherein the 24P converter unit outputs an HD video signal at 24 frames per second, and the pixel interpolation unit generates and outputs a 4K video signal at 24 frames per second from the output of the 24P converter unit."

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, conventional video conversion systems, especially when using certain CMOS image sensors, sometimes produced a rolling shutter effect that caused noticeable distortion of fast-moving subjects due to the lag in pixel readout time caused by the reduction in frame rate. Furthermore, for example, discarding 3 out of 5 frames in a 60p video signal resulted in a loss of smoothness in motion capture compared to shooting at 24p. Therefore, the present invention aims to obtain smooth-moving low-frame-rate video when converting a high-frame-rate video signal to generate a low-frame-rate video signal. [Means for solving the problem]

[0005] To solve the above problems, one representative video signal conversion device of the present invention includes a video conversion unit that converts a first video signal with a high frame rate into a second video signal with a low frame rate, and the video conversion unit is characterized in that it adds up a plurality of consecutive first frames of the first video signal to generate a second frame included in the second video signal. [Effects of the Invention]

[0006] According to the present invention, when converting a high-frame-rate video signal to generate a low-frame-rate video signal, it is possible to obtain a low-frame-rate video with smooth motion. Issues, structures, and effects other than those mentioned above will be clarified by the following explanation of the implementation methods. [Brief explanation of the drawing]

[0007] [Figure 1] A diagram showing an example of the configuration of a video signal conversion device according to an embodiment. [Figure 2] A diagram showing an example of the configuration of the resolution conversion unit according to the embodiment. [Figure 3] A diagram showing the timing chart for frame-based read and write operations in the main memory. [Figure 4]A diagram showing the line-by-line timing chart in the main memory section. [Modes for carrying out the invention]

[0008] When there are multiple components with the same or similar function, they may be described using the same symbol but with different subscripts. Furthermore, if it is not necessary to distinguish between these multiple components, the subscripts may be omitted in the description. The positions, sizes, shapes, and ranges of the components shown in the drawings may not represent their actual positions, sizes, shapes, and ranges in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the positions, sizes, shapes, and ranges disclosed in the drawings.

[0009] [Embodiment] First, with reference to Figure 1, the video signal conversion device 10 according to this embodiment will be described. Figure 1 shows an example of the configuration of the video signal conversion device 10 according to this embodiment. The video signal converter 10 converts, for example, video with a frame rate of 60p used in sports broadcasts to video with a frame rate of 24p used in movies, dramas, etc. The video signal conversion device 10 mainly includes a first data transfer unit 110, a video conversion unit 120, a main memory unit 130, and a resolution conversion unit 140.

[0010] <First Data Transfer Section> The first data transfer unit 110 converts the first video signal 210 into a data transfer format suitable for the video conversion unit 120. The first video signal 210 is a high-resolution and high-frame-rate video signal with a resolution of 4K and a frame rate of 60p. The first video signal 210 is transmitted between video equipment using a data transfer standard suitable for high resolution, such as 12G-SDI. In this embodiment, the first video signal 210 is input to the video signal conversion device 10 in accordance with the 12G-SDI standard. The first data transfer unit 110 performs serial-parallel conversion to convert the input serial format into a parallel format suitable for data transfer to the video conversion unit 120. In this embodiment, the first data transfer unit 110 performs serial-parallel conversion, but may perform conversion to another data transfer standard based on the input data transfer standard. For example, parallel-serial conversion may be performed.

[0011] <Video conversion unit> The video conversion unit 120 converts the first video signal 210 with high resolution and high frame rate into a second video signal 220 with high resolution and low frame rate. The video conversion unit 120 transmits the converted second video signal 220 to the resolution conversion unit 140 that converts it into a third video signal 230 suitable for output to an external device. At this time, the video conversion unit 120 converts it into the second video signal 220 without changing the resolution of the first video signal 210. Details of the processing will be described later with reference to FIG. 3.

[0012] <Main memory unit> The main memory unit 130 is directly connected to the video conversion unit 120 and temporarily stores data for the video conversion unit 120 to execute processing. Note that since the main memory unit 130 is well-known, the description of its specific configuration is omitted. In this embodiment, the main memory unit 130 is a DRAM conforming to the DDR4 SDRAM standard, but other standards may also be used. For example, it may be DDR5 SDRAM or LPDDR.

[0013] <Resolution conversion unit> Next, referring to FIG. 2, the resolution conversion unit 140 according to the embodiment will be described. FIG. 2 is a diagram showing an example of the configuration of the resolution conversion unit 140 according to this embodiment. The resolution conversion unit 140 converts the video signal into a format suitable for the external device to play back the video signal, and transfers the video signal to the external device. Specifically, the resolution conversion unit 140 converts the second video signal 220 into a third video signal 230. The third video signal 230 may include multiple types of formats. Here, the third video signal 230 is composed of third frames that constitute a video signal with a low frame rate. The resolution conversion unit 140, for example, converts the resolution of the second video signal 220 from 4K to 2K. The resolution conversion unit 140 can, for example, convert only the upper left video obtained by dividing the video of the second video signal 220 into four parts. The resolution conversion unit 140 can, for example, convert the second video signal 220 into a third video signal 230 suitable for output in 6G-SDI. The resolution conversion unit 140 mainly includes a pixel alignment unit 141, a pixel conversion unit 142, and a second data transfer unit 143.

[0014] The pixel alignment unit 141 rearranges the pixel array of the second video signal 220 from a 2-sample interleave array to a square division array. The pixel alignment unit 141, for example, executes a process of making a four-divided video when dividing the video recorded in the first video signal 210 into four parts and outputting it. If the pixel alignment unit 141 does not require the process, it may not execute the process and output the second video signal 220 as it is.

[0015] The pixel conversion unit 142 converts the resolution of the second video signal 220. In this embodiment, the conversion of the frame rate is performed by the video conversion unit 120, and the conversion of the resolution is performed by the pixel conversion unit 142, but it is not necessarily limited to this. For example, the conversion may be performed integrally. The pixel conversion unit 142 converts to a resolution suitable for the output of the external device. For example, the pixel conversion unit 142 converts the resolution from 4K to 2K while keeping the frame rate unchanged and outputs it.

[0016] The pixel conversion unit 142 may convert the resolution of only a specific portion when converting the resolution. For example, it may convert and output the resolution of only the upper left portion when the video recorded on the first video signal 210 is divided into four sections. If the pixel conversion unit 142 does not need to perform the processing, it may not perform the processing and may output the second video signal 220 as is.

[0017] The second data transfer unit 143 converts the second video signal 220 into a data transfer format suitable for external output. In this embodiment, the third video signal 230 is output to an external device, for example, in the 12G-SDI standard. The second data transfer unit 143 performs a parallel-to-serial conversion to convert the input parallel format into a serial format suitable for data transfer to an external device. In this embodiment, the second data transfer unit 143 performs parallel-to-serial conversion, but it may perform conversion to another data transfer standard based on the input data transfer standard. For example, it may perform serial-to-parallel conversion.

[0018] Next, the processing of the video conversion unit 120 will be explained with reference to Figure 3. Figure 3 is a timing chart showing the frame-by-frame read and write operations in the main memory unit 130.

[0019] When the video conversion unit 120 converts a high-frame-rate video signal to a low-frame-rate video signal, it adds up a number of consecutive first frames to generate a second frame. As shown in Figure 3, the video conversion unit 120 converts the first video signal 210 with a frame rate of 60p to a second video signal 220 with a frame rate of 24p. Here, the first frame is a frame that constitutes a high-frame-rate video signal. The second frame is a frame that constitutes a low-frame-rate video signal.

[0020] In the following process, we will explain assuming that, as shown in this embodiment, the first video signal 210 is a video with a resolution of 4K, a frame rate of 60p, and a duration of 0.2 seconds, the second video signal 220 is a video with a resolution of 4K, a frame rate of 24p, and a duration of 0.2 seconds, and the third video signal 230 is a video with a resolution of 2K, a frame rate of 24p, and a duration of 0.2 seconds. However, video signals with other resolutions, frame rates, and durations may also be used.

[0021] In carrying out the invention, the frame rate of the first video signal 210 may be any other frame rate as long as it is higher than the frame rate of the second video signal 220. For example, the frame rate of the first video signal 210 may be 100p, or the frame rate of the second video signal 220 may be 12p.

[0022] If the frames included in the first video signal 210 are numbered sequentially from the beginning, the first video signal 210 will contain the first frames, with frame numbers from Fin0 to Fin11. Similarly, if the frames included in the second video signal 220 are numbered sequentially from the beginning, the second video signal 220 will contain the second frames, with frame numbers from Fmi0 to Fmi4.

[0023] In this embodiment, when the video conversion unit 120 converts the first video signal 210 with a frame rate of 60p to the second video signal 220 with a frame rate of 24p, the number of frames is reduced by 36. In other words, the number of frames is reduced to 1 / 2.5 times.

[0024] First, the video conversion unit 120 writes all of the first frames of the first video signal 210 to the main memory unit 130. Next, the video conversion unit 120 reads the first frame from the main memory unit 130 in multiple timings.

[0025] For example, the video conversion unit 120 reads the first frame with frame numbers Fin0, Fin2, Fin5, and Fin7 at three different timings: (1) when reading the first frame with frame numbers Fin1, Fin3, Fin6, and Fin8 at two different timings (2) when reading the first frame with frame numbers Fin4 and Fin9 at three different timings (3).

[0026] Here, the process by which the video conversion unit 120 writes all first frames of the first video signal 210 to the main memory unit 130 and the process by which the video conversion unit 120 reads the first frames from the main memory unit 130 are executed simultaneously, but the first frames with the same frame number are not written and read simultaneously. For example, the first frame with frame numbers Fin0 and Fin1 is written to the main memory 130, and then read from the main memory 130 when the first frame with frame number Fin2 is written. At this time, the first frame with frame numbers Fin0 and Fin1 is read with a delay of approximately 2 frames.

[0027] Next, the video conversion unit 120 generates a second frame by combining and adding two or more consecutive frames whose frame numbers were read at different timings. In Figure 3, the lower row shows the second frame generated based on the first frame that was read.

[0028] For example, the video conversion unit 120 generates the second frame with frame number Fmi0 by combining it with the first frame with frame number Fin0 and frame number Fin1. For example, the video conversion unit 120 generates the second frame with frame number Fmi1 by combining the first frames Fin2, Fin3, and Fin4. For example, the video conversion unit 120 generates the second frame with frame number Fmi2 by combining the first frames Fin7, Fin8, and Fin9.

[0029] The video conversion unit 120 generates the second frame by equally adding the first frame, but it may also generate it using weighting. For example, when the video conversion unit 120 generates the second frame with frame number Fmi1, it may add Fin2 at a ratio of 1 / 4, Fin3 at 1 / 4, and Fin4 at 1 / 2.

[0030] In this embodiment, the video conversion unit 120 reads all of the first frames of the written first video signal 210 in three separate timings, but it may also read in a number of times other than three.

[0031] Next, referring to Figure 4, the processing performed by the video conversion unit 120 on a line-by-line basis will be explained. Figure 4 shows a line-by-line timing chart for the main memory unit 130. In the following process, we will describe the timing chart of the main memory unit 130 in area A of Figure 3, but the same applies to other areas.

[0032] Figure 4(A) is a chart showing the synchronization signals when the first video signal 210 is being written. Figure 4(B) is a timing chart showing the writing process of the first frame on a line-by-line basis. Figure 4(C) is a timing chart showing the line-by-line reading process of the first frame for generating the second frame. Here, the hexagonal symbol with a solid line and the letter n inside, shown in Figure 4(B), represents the nth line of the first frame of Fin4 (where n is a positive integer, and so on). Here, the circle symbol with "n" written inside, shown in Figure 4(C), indicates the nth line of the first frame of Fin2. Here, the triangular symbol with "n" written inside, shown in Figure 4(C), represents the n line of the first frame of Fin3. Here, the rectangular symbol with the letter n inside, shown in Figure 4(C), represents the n-line of the first frame of Fin4.

[0033] The video conversion unit 120 writes the first frame with frame number Fin4 based on the synchronization signal shown in Figure 4(A). For example, the video conversion unit 120 writes the first line of the first frame of Fin4 to the main memory unit 130 based on the synchronization signal 301. Similarly, the second line of the first frame of Fin4 is written to the main memory 130 based on the synchronization signal 302, the third line of the first frame of Fin4 is written based on the synchronization signal 303, the fourth line of the first frame of Fin4 is written based on the synchronization signal 304, and the fifth line of the first frame of Fin4 is written based on the synchronization signal 305. Similarly, the video conversion unit 120 writes all lines of the first frame of Fin4 to the main memory unit 130 based on the synchronization signal.

[0034] Next, the video conversion unit 120 reads from the main memory unit 130 the line of the first frame that will be used to generate the second frame with frame number Fmi1. For example, as shown in Figure 3, when writing the first frame with frame number Fin4, the video conversion unit 120 simultaneously performs the reading process of the first frames with frame numbers Fin2, Fin3, and Fin4, which will be used to generate the second frame with frame number Fmi1.

[0035] At this time, as shown in Figure 4, the video conversion unit 120 generates the first line of the second frame with frame number Fmi1 at the timing when the synchronization signal 301 is emitted, and reads the first line of the first frame with frame numbers Fin2, Fin3, and Fin4 from the main memory unit 130.

[0036] Next, as shown in Figure 4, the video conversion unit 120 generates the second line of the second frame with frame number Fmi1 at the timing when the synchronization signal 303 is emitted, and reads the second line of the first frame with frame numbers Fin2, Fin3, and Fin4 from the main memory unit 130.

[0037] Similarly, the video conversion unit 120 generates the corresponding lines of the second frame of Fmi1 based on some of the synchronization signals, and reads the corresponding lines of the first frame with frame numbers Fin2, Fin3, and Fin4 from the main memory unit 130.

[0038] Furthermore, in this embodiment, the video conversion unit 120 is described as reading the corresponding lines of the first frame with frame numbers Fin2, Fin3, and Fin4 based on a synchronization signal, but this process does not necessarily have to be based on a synchronization signal. For example, the video conversion unit 120 may perform the read operation independently of the write operation, provided that it does not interfere with the read operation of the first frame used to generate the second frame.

[0039] In this embodiment, the first frame included in the first video signal 210 can be used to generate the second frame that constitutes the output second video signal 220 without discarding it. As a result, even if the second video signal 220 has a lower frame rate than the first video signal 210, the smoothness of the video motion can be maintained.

[0040] The present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention.

[0041] Furthermore, the present invention can also take the following forms. (Aspect 1) It includes a video conversion unit that converts a first video signal with a high frame rate into a second video signal with a low frame rate. The video conversion unit adds up a plurality of consecutive first frames of the first video signal to generate a second frame included in the second video signal. A video signal conversion device characterized by the following features. (Aspect 2) A video signal conversion device according to Embodiment 1, The video conversion unit generates the second frame by weighting and adding each of the first frames. A video signal conversion device characterized by the following: (Aspect 3) A video signal conversion device according to embodiment 1 or 2, The video conversion unit adds two or more consecutive first frames. A video signal conversion device characterized by the following features. (Aspect 4) A video signal conversion device according to any one of embodiments 1 to 3, The system includes a main memory unit directly connected to the aforementioned video conversion unit. A video signal conversion device characterized by the following: (Appendix 5) A video signal conversion device according to any one of embodiments 1 to 4, The system includes a resolution conversion unit that converts the second video signal into a third video signal with a different resolution from the first video signal. A video signal conversion device characterized by the following: [Explanation of symbols]

[0042] 10. Video signal converter 110 First data transfer unit 120 Video Conversion Unit 130 Main memory 140 Resolution conversion section 141 Pixel alignment section 142 Pixel Conversion Section 143 Second data transfer section 210 First video signal 220 Second video signal 230 Third video signal 301, 302, 303, 304, 305 Synchronization signals

Claims

1. It includes a video conversion unit that converts a first video signal with a high frame rate into a second video signal with a low frame rate. The video conversion unit adds up a plurality of consecutive first frames of the first video signal to generate a second frame included in the second video signal. A synchronization signal is emitted at a predetermined interval to perform a write operation for each line of the first frame, and, Within the predetermined period of a single synchronization signal, after performing a write operation on one line of the last first frame Z that is the target of the write operation among the multiple consecutive first frames, a read operation is performed on one line of the first frame Z and the first frame preceding the first frame Z. A video signal conversion device characterized by the following:

2. A video signal conversion device according to claim 1, The read operation for the next line of the first frame Z is performed within the predetermined period of the synchronization signal that is emitted after the first synchronization signal. A video signal conversion device characterized by the following:

3. A video signal conversion device according to claim 1, The video conversion unit generates the second frame by weighting and adding each of the first frames. A video signal conversion device characterized by the following:

4. A video signal conversion device according to claim 1, The video conversion unit adds two or more consecutive first frames. A video signal conversion device characterized by the following features.

5. A video signal conversion device according to claim 1, The system includes a main memory unit directly connected to the aforementioned video conversion unit. A video signal conversion device characterized by the following:

6. A video signal conversion device according to claim 1, The system includes a resolution conversion unit that converts the second video signal into a third video signal with a different resolution from the first video signal. A video signal conversion device characterized by the following:

Images