Waveform monitor and display generation method

Abstract

To allow for compact display of a wide dynamic range, by using a familiar number of aperture (F) stages.SOLUTION: A waveform monitor 20 generates a modified image from an original image of a camera 12. A brightness measuring system of the waveform monitor 20 generates brightness values of pixels of the original image and then converts them to equivalent F-stage numerical values. A cursor range selection block 70 creates a range of the F-stage numerical values, and a color cursor mixer 80 selectively colors, with a pseud color, pixels included in the range of the F-stage numerical values in the original image and modifies them.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for displaying information on a waveform monitor, and more particularly, to a waveform monitor for monitoring the output signal waveform of a camera and a method for generating its display.

Background Art

[0002] In video production and movie production at a shooting site, it is often necessary to use an incident light meter or a reflected light (spot) meter to adjust the lighting of a shooting scene, the gain of a camera, or the aperture. Measurements and lighting adjustments using such an exposure meter (light meter) are performed with relative values such as aperture steps (F-stops). As is well known to users, the user changes the aperture step in the process of adjusting the combination of the exposure time (shutter speed) of the imaging device and the aperture value (F-value) of the lens. The F-value is a dimensionless ratio derived by dividing the focal length of the lens by the effective aperture. For example, when the aperture of the lens is opened by one step (for example, the F-value is decreased from 2.8 to 2), the amount of light passing through the lens increases by a factor of 2 or approximately 6 dB. Conversely, when the aperture of the lens is closed by one step (for example, the F-value is increased from 2 to 2.8), the amount of light passing through the lens is halved. However, even when the F-value is changed by one step, the F-value changes only at a ratio of the square root of 2 (= 1.4142 ···) (see Non-Patent Document 1). The F-value typically functions as an index for adjusting the aperture (opening) of the lens.

[0003] Currently, in both movie production and video production, cameras using electronic image sensors are utilized. These image sensors have a very large dynamic range and a very large variable gain. For example, in the case of a typical digital single-lens reflex camera (e.g., Nikon D800), it has 14.4 stops, while in the case of negative film, it has 13 stops (about 6 dB / stop). In movie production and video production, not only when compressing to a 12-bit resolution digital video output signal, but even when compressing to a 10-bit resolution digital video output signal, in addition to relatively new logarithmic (log) processing in the camera, traditional gamma (power-law) correction still occupies a significant position regarding the dynamic range. Note that as a recent trend, digital single-lens reflex cameras are used not only for still images but also for video production. According to this, it is possible to produce videos with significant blurring at a much lower cost compared to conventional professional video equipment.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] During shooting and the like, it is very important to determine how to make good use of the dynamic range based on camera adjustments (gain and aperture) and shooting scene lighting. This is typically done by observing the camera's output signal on a picture monitor (with a dynamic range of less than 10 steps). At this time, even when using a well-calibrated picture monitor, if simply observed, due to eye adaptation, detailed parts will sink into black and cannot be recognized, and it should be noted that the dynamic range will be limited to about 7 steps. Video waveform monitors are also often used, but currently, such waveform monitors are limited to linear voltage displays, and most of the dynamic range is close to black compressed to a few mV close to 0. Similar to the case of analyzing high dynamic range radio frequency (RF) signals, linear waveform scaling is not appropriate.

[0007] Embodiments of the present invention attempt to solve such problems of the prior art and other problems.

Means for Solving the Problems

[0008] Embodiments of the present invention include a method and apparatus for observing the weighted output signal waveform of a camera in real time, which combines gamma correction and other non-linear corrections applied to the camera's imaging device, and converts the waveform into a waveform weighted by the "F number" of log2 and displays it together with a calibrated scale and a cursor.

[0009] In addition, embodiments of the present invention include a method and apparatus for making any region of the calibrated F-number weighted waveform selectable using a variable marker or cursor that can read out the F-number value. According to these variable markers, preferably, the corresponding region on the image display of the camera image can be indicated corresponding to the F-number value selected by this marker.

[0010] As an embodiment of the present invention, there is also a waveform monitor that generates a modified image from an original image. This waveform monitor includes a luminance measurement system that generates luminance values and subsequently converts them into equivalent F-stop values. A selection unit is used to select a range of F-stop values. An image modification unit modifies the original image with respect to the selected pixels that fall within the selected range of F-stop values. For example, with respect to the original image, the selected pixels may be replaced with or mixed with colored pixels, that is, the original image may be modified by pseudo-coloring the selected pixels of the original image (changing the original color to a substitute color). A method for modifying an image is also disclosed in such a manner.

[0011] Viewing the present invention from several perspectives, Concept 1 of the present invention is a method for generating a display regarding an output signal of a measuring device, comprising: a process of receiving an original image composed of a plurality of pixels; a process of extracting luminance information from the pixels; a process of converting the luminance information of the pixels into equivalent F-stop values (f-stop equivalent); a process of selecting equivalent F-stop values that fall within a first range of F-stop values; a process of modifying the original image by modifying the original pixels that fall within the first range of the F-stop values to generate a modified image; a process of displaying the modified image and comprising.

[0012] Concept 2 of the present invention is the display generation method of Concept 1, wherein the width of the first range of the F-stop values is adjustable by a user.

[0013] Concept 3 of the present invention is the display generation method of Concept 1, wherein the center of the first range of the F-stop values is adjustable by a user.

[0014] Concept 4 of the present invention is the display generation method of Concept 1 above. At this time, the process of changing the original image by changing the original pixels includes a process of replacing at least a part of the original pixels falling within the first range of the F-stage value with color pixels.

[0015] Concept 5 of the present invention is the display generation method of Concept 1 above. At this time, the process of changing the original image by changing the original pixels includes a process of combining the luminance values of at least some of the original pixels falling within the first range of the F-stage value with color values.

[0016] Concept 6 of the present invention is the display generation method of Concept 1 above, a process of selecting equivalent F-stage values falling within a second range of F-stage values, a process of changing the original image by changing the original pixels falling within the second range of the F-stage value to generate a changed image, and further includes, At this time, the process of changing the original pixels falling within the second range of the F-stage value is different from the process of changing the original pixels falling within the first range of the F-stage value. That is, the first range and the second range are different, and different pixels are changed in each process.

[0017] Concept 7 of the present invention is the display generation method of Concept 6 above. At this time, the process of changing the original pixels falling within the first range of the F-stage value includes a process of changing the original pixels by applying a first color to the original pixels, and the process of changing the original pixels falling within the second range of the F-stage value includes a process of changing the original pixels by applying a second color to the original pixels.

[0018] Concept 8 of the present invention is a method of generating a display related to the output signal of a measuring device, a process of receiving an original image composed of a plurality of pixels, A process for obtaining the equivalent f-stop value of the above pixel, and A process for generating a modified image by pseudo-coloring a part of the original image based on the equivalent f-stop value, and A process for displaying the modified image and It comprises.

[0019] Concept 9 of the present invention is the display generation method of Concept 8 above. At this time, the process of pseudo-coloring a part of the original image includes a process of replacing at least a part of the original pixels falling within the first range of the f-stop value with color pixels.

[0020] Concept 10 of the present invention is the display generation method of Concept 8 above. At this time, the process of pseudo-coloring a part of the original image includes a process of mixing at least a part of the original pixels falling within the first range of the f-stop value with color pixels.

[0021] Concept 11 of the present invention is a waveform monitor having an image input unit and a monitor for viewing the display of measurement values, A measurement system for measuring the luminance value of the pixels of the original image received by the image input unit, A conversion unit configured to generate an f-stop value from the luminance value, A selection unit configured to generate a range of f-stop values and A modification unit configured to modify the original image with respect to the selected pixels falling within the range of the f-stop value and It comprises.

[0022] Concept 12 of the present invention is the waveform monitor of Concept 11 above. At this time, the modification unit is configured to replace the selected pixels with color pixels.

[0023] Concept 13 of the present invention is the waveform monitor of Concept 11 above. At this time, the modification unit is configured to combine the selected pixels with color pixels.

[0024] Concept 14 of the present invention is the waveform monitor of Concept 11, wherein the selection unit is controllable by the user.

[0025] Concept 15 of the present invention is the waveform monitor of Concept 11, wherein the selection unit is further configured to generate a range of second F-step values.

[0026] Concept 16 of the present invention is the waveform monitor of Concept 15, wherein the modification unit is further configured to modify the original image with respect to the selected pixels falling within the range of the second F-step values.

[0027] Concept 17 of the present invention is the waveform monitor of Concept 11, further comprising an output signal generation structure for generating a time-versus-F-step waveform.

[0028] Concept 18 of the present invention is the waveform monitor of Concept 11, wherein the converter calibrated to generate the F-step value has a look-up table (LUT).

[0029] Concept 19 of the present invention is the waveform monitor of Concept 18, wherein the LUT is one of a plurality of pre-stored LUTs accessible by the waveform monitor.

[0030] Concept 20 of the present invention is the waveform monitor of Concept 17, wherein the monitor displays the time-versus-F-step waveform and a user-controllable cursor for defining the range of the F-step value.

Brief Description of the Drawings

[0031]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0032] As described above, the embodiments of the present invention can be used to evaluate live video signals from a camera from the perspective of relative aperture steps (log2 scale: the F value changes by 1 step every time the amount of light changes by a factor of 2), in addition to traditional linear voltage displays and IRE levels. This effectively converts the output signal of an analog camera, even, in terms of the aperture step (f-stop), into the relative lighting and exposure values of an exposure meter (light meter). Note that IRE (Institute of Radio Engineers) is a unit when expressing the pedestal level of a composite video signal of video as 0% and the full white level as 100%. 100 IRE corresponds to 714 mV in NTSC and 700 mV in PAL.

[0033] FIG. 1 is a block diagram centered on the hardware part of an example of a video waveform monitor according to the present invention. As shown in FIG. 1, a waveform monitor 20 is connected to a camera 12 and receives an input signal. The camera 12 is directed at a subject 14, and the subject 14 is illuminated by a lighting 16. The camera 12 usually has an aperture and exposure index adjustment mechanism, which may be controlled by the operator of the camera or may be automatically adjusted by the camera 12. The output signal of the camera 12 is input to the waveform monitor 20.

[0034] The output signal of camera 12 is first processed by the input processor 30 and then passes through a Luma (luminance) filter 32. The filter 32 is, for example, a low-pass filter, and the user may be able to turn the filter 32 on / off using a user interface (UI) 50 or the like. The filtered output signal or the unfiltered output signal is supplied to a look-up table (LUT) 40. The LUT 40 is loaded via the user interface 50 using a preset table (preset table). Several preset tables may be prepared in advance, and the user may be able to select and load one of them. For example, as one option of the LUT 40, a LUT 40A for performing gamma correction (gamma removal) or logarithmic (log) processing on the luma signal (luminance signal) may be used. As another option of the LUT 40, there is a LUT 40B that converts the luma signal Y to the Log2(Y / Ymax) scale, thereby enabling real-time waveform display of the F-step luma signal (such a waveform display is shown in FIG. 2B). At this time, Ymax is the maximum value of the luma signal (luminance signal), that is, the white level (700 mV in PAL). Since the logarithm to the base 2 is used, when (Y / Ymax) doubles, the F-step value increases by 1. For example, in this example, the F-step value -2.4 corresponds to 132.6 mV when converted to voltage, and the F-step value -5.9 corresponds to 11.7 mV. Thus, in the display examples shown in FIGS. 2A and 2B, the cursors 1 and 2 are in the same positions, but the difference (delta) between the cursors 1 and 2 in FIG. 2B is significantly larger when considered in terms of the voltage conversion compared to the case of displaying on a linear voltage scale as shown in FIG. 2A. It can also be seen that the vicinity of the black level (0 voltage) can be displayed in more detail compared to the linear voltage scale. To reduce the size of the LUT, both multiplication and logarithmic scaling arithmetic operations are accurately performed in advance, and all words of the LUT are converted into a single data set, so a large word size is not required to represent a wide dynamic range of linear light intensity values.

[0035] As described above, there may be a plurality of LUTs stored in the waveform monitor 20. The user can control which of the plurality of stored LUTs is loaded as the active LUT 40 using the user interface 50. For example, various LUTs may be stored in the waveform monitor 20 in advance, so that the user can select the active LUT 40 based on the gamma or black level of the camera.

[0036] The display monitor (display device) 60 on the waveform monitor 20 displays the output signal to the user. The monitor 60 can be used to display a traditional time-versus-voltage waveform as shown in FIG. 2A, while it can also be used to show a new time-versus-F-step waveform as shown in FIG. 2B. In this new time-versus-F-step waveform display, similar to the traditional time-versus-voltage waveform display, the user can utilize an adjustable cursor, but the scale of the output signal is "F-step" instead of voltage. As shown in FIGS. 2A and 2B, the horizontal time axis is the same in both displays, and conventional scales such as 1 line, 2 lines, field sweep, etc. can be selected.

[0037] In addition, the cursor range (window) selection block 70 receives an input from the user through the user interface 50 and reads a plurality of variable cursor values set by the user. These variable cursor values are used as binary gate signals to modify the monochrome output signal of the waveform monitor 20. Specifically, the input processor 30 receives the input signal from the camera and performs necessary processing, and the color cursor mixer 80 is coupled to the input processor 30 and receives the processed signal from the input processor 30. Note that the color cursor mixer 80 may be configured to receive the input signal processed from another component of the waveform monitor 20. The color cursor mixer 80 is also coupled to the cursor range selection block 70. The binary gate signal from the cursor range selection block 70 is used to determine which region of the original monochrome output signal is to be colored, i.e., the region to be highlighted in the output signal. In this way, the color cursor mixer 80 functions as an image modification unit.

[0038] For example, when comparing the output signals of FIGS. 3 and 6, FIG. 3 is the original monochrome output signal, while the output signal of FIG. 6 has a portion highlighted in color to indicate the image area within the F-step range selected by the user on the original monochrome image. The captured image shown in FIG. 3 is only luma (brightness), that is, a monochrome image, while in FIG. 6, as will be described in detail later, a cursor range (for example, red and blue) by two colors is added. Note that in FIG. 6, since the drawing is in grayscale, the red area is shown as a horizontal stripe pattern and the blue area is shown as a vertical stripe pattern (such as the lower part of a person's chin).

[0039] Referring to FIG. 1 again, to generate the output signal of FIG. 6, in the detection circuit within the cursor range selection block 70, the log2(Y / Ymax) signal from the LUT 40 (specifically, LUT 40B) is compared with two cursor ranges controllable by the user. The user can control the position and size of the cursor range. Specifically, in one embodiment, the user may control the center value of the F-step value and the size of the cursor range. In another embodiment, the user may define the cursor range using two cursors. In the system, a binary gate signal is generated using this range. When the F-step value (brightness value) of a pixel in the luma output signal falls within the specified cursor range, that pixel is shown as a colored pixel in the display, thereby providing the user with information on which pixels in the output signal are within the specified F-step range. The remaining pixels, that is, the pixels having a luminance level outside the specified cursor range in the original pixels, are shown without change in the display.

[0040] The cursor range may be adjustable, for example, in steps of 1 / 4 stop. Also, for example, the cursor range may be preset to a range of plus or minus 1 / 4 stop from the F-stop center value set by the user. The F-stop center value is adjustable across the entire range of the signal obtained from the LUT 40. In this way, the user can adjust one cursor to highlight a specific area of the image, and thereby, in addition to the specific area on the image having the luminance value of that F-stop value from the value of that cursor on the F-stop waveform shown in FIG. 2B or FIG. 5, since there is a width of plus or minus 1 / 4 stop, other areas having luminance values of approximately the same F-stop value can be identified.

[0041] The color cursor mixer 80 generates, for example, coloring signals for red and blue pixels, and synthesizes them with the original image to generate a modified image as shown in FIG. 6. In one embodiment, the color cursor mixer 80 simply replaces, for example, the original pixels falling within the set F-stop range with single-color red or blue pixels. In another embodiment, the color cursor mixer 80 may generate a color-mixed output signal by adding a color tone to the underlying luminance data. Also, in these examples, a monochrome-based image is shown, but embodiments of the present invention are not limited to luminance-based ones, and may be implemented independently for each of the red, green, and blue color channels, for example.

[0042] In the example shown in FIG. 6, there are two-color cursor ranges, but the waveform monitor 20 can generate more cursor ranges or fewer cursor ranges.

[0043] FIG. 4 shows the traditional voltage waveform (VmV_wfm n ) of the image of FIG. 3. FIG. 5 similarly shows the output signal for the image of FIG. 3, but in FIG. 5, it shows the F-stop waveform (Fstop_wfm n ) for the image of FIG. 3 according to an embodiment of the present invention, having two cursors, FS low (meaning low F-stop) and FS high (meaning high F-stop).

[0044] By using the embodiments of the present invention, in conjunction with the lighting of the shooting scene, the adjustment of the camera's gain / aperture / shutter speed can be easily performed in familiar F-stop units, and the camera can be effectively used as an exposure meter. For example, by using the embodiments of the present invention, as shown in FIG. 5, the user can view a waveform weighted by the F-stop on a display showing graduations of a linearized scale in F-stop units. Further, according to the embodiments of the present invention, by enabling the user to control the differential measurement value of the delta F-stop of the content elements of the image / scene by using the F-stop waveform and the pseudo-color region on the monochrome image display, the user can measure the hot spots and lighting non-uniformity of the scene in F-stop units using two cursors. Furthermore, according to the embodiments of the present invention, the characteristics of the camera (camera matching) can be adjusted by performing accurate black level adjustment and high-resolution black balance adjustment (adjusting the levels of the three primary RGB colors to accurately reproduce black), and the noise of the camera can be displayed with high resolution. Furthermore, the embodiments of the present invention can be used as a tool for evaluating the dynamic range of the content of a video scene, and thereby can be used to optimize the dynamic range from the perspective of artistic quality (video expression).

[0045] The waveform monitor 20 and any of its components may be implemented in firmware in a circuit or FPGA designed for a specific purpose such as an ASIC, or may be implemented as one or more software processes executed on one or more processors. In another embodiment, the waveform monitor 20 may include, for example, a combination of components or processes executed on firmware, ASIC, FPGA, and software.

[0046] Specific embodiments of the present invention have been described by way of examples and for the purpose of illustration, but it will be apparent that various modifications are possible without departing from the spirit and scope of the present invention.

Description of Reference Numerals

[0047] 12 Camera 14 Subject 16 Lighting 20 Waveform Monitor 32 Luma Filter (Brightness Filter) 40 F-stop Lookup Table (Conversion Unit) 50 User Interface 60 Display Device (Monitor) 70 Cursor Range Selection Block (Selection Unit) 80 Color Cursor Mixer (Image Modification Unit)

Claims

Claim 1 A waveform monitor having an image input unit and a monitor for viewing the display of measurement values, a measurement system for measuring the luminance value of pixels from the original image received by the image input unit, a conversion unit configured to perform gamma removal or logarithmic processing on the measured luminance value, and further convert the luminance value subjected to the gamma removal or logarithmic processing into an equivalent F-number Log2(Y / Ymax) scale using a look-up table to generate an equivalent F-number value from the measured luminance value, where Y is the luminance value of the measured luminance value and Ymax is the maximum luminance value among the measured luminance values, a selection unit configured to generate a range of first F-number values, a color mixer configured to make the selected pixels of the original image that fall within the range of the first F-number values into colored pixels and pass the monochrome pixels of the unselected pixels of the original image as they are, and an output signal generation structure configured to display the image changed from the original image by the color mixer A waveform monitor comprising.

Images