WAVEFORM MONITOR AND DISPLAY GENERATION METHOD - Patent application

Abstract

To allow for compact display of a wide dynamic range, by using a familiar number of aperture (F) stages.SOLUTION: A waveform monitor 20 generates a modified image from an original image of a camera 12. A brightness measuring system of the waveform monitor 20 generates brightness values of pixels of the original image and then converts them to equivalent F-stage numerical values. A cursor range selection block 70 creates a range of the F-stage numerical values, and a color cursor mixer 80 selectively colors, with a pseud color, pixels included in the range of the F-stage numerical values in the original image and modifies them.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for displaying information on a waveform monitor, and more particularly, to a waveform monitor for monitoring the output signal waveform of a camera and a method for generating its display.

Background Art

[0002] In video production and movie production at a shooting site, it is often necessary to use an incident light meter or a reflected light (spot) meter to adjust the lighting of a shooting scene, the gain of a camera, or the aperture. Measurements and lighting adjustments using such an exposure meter (light meter) are performed using relative values such as aperture steps (F-stops). As is well known to users, the user changes the aperture step in the process of adjusting the combination of the exposure time (shutter speed) of the image sensor and the aperture value (F-value) of the lens. The F-value is a dimensionless ratio derived by dividing the focal length of the lens by the effective aperture. For example, when the aperture of the lens is opened by one step (for example, the F-value is reduced from 2.8 to 2), the amount of light passing through the lens increases by a factor of 2 or approximately 6 dB. Conversely, when the aperture of the lens is closed by one step (for example, the F-value is increased from 2 to 2.8), the amount of light passing through the lens is halved. However, even when the F-value is changed by one step, the F-value changes only at a ratio of the square root of 2 (= 1.4142 ···) (see Non-Patent Document 1). The F-value typically functions as an indicator for adjusting the aperture (opening) of the lens.

[0003] Currently, in both movie production and video production, cameras using electronic image sensors are utilized. These image sensors have a very large dynamic range and a very large variable gain. For example, compared to 14.4 stops in a typical digital single-lens reflex camera (such as the Nikon D800), negative film has 13 stops (about 6 dB / stop). In movie production and video production, in addition to compressing to a 12-bit resolution digital video output signal, even when compressing to a 10-bit resolution digital video output signal, traditional gamma (power-law) correction, in addition to relatively new logarithmic (log) processing in the camera, still occupies a large position regarding the dynamic range. Note that as a recent trend, digital single-lens reflex cameras are being used not only for still images but also for video production. According to this, it is possible to produce videos with significant blurring while being significantly cheaper compared to conventional professional video equipment.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] During shooting, etc., it is very important to determine how to make good use of the dynamic range based on camera adjustments (gain and aperture) and shooting scene lighting. Typically, this is done by observing the camera's output signal on a picture monitor (with a dynamic range of less than 10 stops). At this time, even when using a well-calibrated picture monitor, if simply observed, due to eye adaptation, detailed parts will sink into black and cannot be recognized, and it should be noted that the dynamic range will be limited to about 7 stops. Video waveform monitors are also often used, but currently, such waveform monitors are limited to linear voltage displays, and most of the dynamic range is close to black, compressed to just a few mV close to 0. Similar to analyzing high dynamic range radio frequency (RF) signals, linear waveform scaling is not appropriate.

[0007] Embodiments of the present invention seek to solve these problems of the prior art and other problems.

Means for Solving the Problems

[0008] Embodiments of the present invention include a method and apparatus for observing the weighted output signal waveform of a camera in real time, which combines gamma correction and other non-linear corrections applied to the camera's image sensor, and displays it together with calibrated scales and a cursor after converting it into a waveform weighted by the "F-stop" of log 2 .

[0009] In addition, embodiments of the present invention include a method and apparatus for making any region of the calibrated F-stop weighted waveform selectable using a variable marker or cursor capable of reading out the F-stop value. According to these variable markers, preferably, the corresponding region on the image display of the camera image can be indicated corresponding to the F-stop value selected by this marker.

[0010] As an embodiment of the present invention, there is also a waveform monitor that generates a modified image from an original image. This waveform monitor includes a luminance measurement system that generates luminance values and subsequently converts them into equivalent F-stop values. A selection unit is used to select a range of F-stop values. An image modification unit modifies the original image with respect to selected pixels that fall within the selected range of F-stop values. For example, with respect to the original image, the selected pixels may be replaced with or mixed with colored pixels, that is, the original image may be modified by pseudo-coloring (changing the original color to a substitute color) the selected pixels of the original image. A method for modifying an image is also disclosed in such a manner.

[0011] Viewing the present invention from several perspectives, Concept 1 of the present invention is a method for generating a display regarding the output signal of a measuring device, comprising a process of receiving an original image composed of a plurality of pixels, a process of extracting luminance information from the above pixels, a process of converting the luminance information of the above pixels into equivalent F-stop values (f-stop equivalent), a process of selecting equivalent F-stop values that fall within a first range of F-stop values, a process of modifying the original image by modifying the original pixels that fall within the above first range of F-stop values to generate a modified image, a process of displaying the above modified image and comprising.

[0012] Concept 2 of the present invention is the display generation method of Concept 1, wherein the width of the above first range of F-stop values is adjustable by a user.

[0013] Concept 3 of the present invention is the display generation method of Concept 1, wherein the center of the above first range of F-stop values is adjustable by a user.

[0014] Concept 4 of the present invention is the display generation method of Concept 1 above. At this time, the process of changing the original image by changing the original pixels includes a process of replacing at least a part of the original pixels falling within the first range of the F-stage value with color pixels.

[0015] Concept 5 of the present invention is the display generation method of Concept 1 above. At this time, the process of changing the original image by changing the original pixels includes a process of combining the luminance values of at least some of the original pixels falling within the first range of the F-stage value with color values.

[0016] Concept 6 of the present invention is the display generation method of Concept 1 above, a process of selecting equivalent F-stage values falling within the second range of the F-stage value, a process of changing the original image by changing the original pixels falling within the second range of the F-stage value to generate a changed image, and further includes, At this time, the process of changing the original pixels falling within the second range of the F-stage value is different from the process of changing the original pixels falling within the first range of the F-stage value. That is, the first range and the second range are different, and different pixels are changed in each process.

[0017] Concept 7 of the present invention is the display generation method of Concept 6 above. At this time, the process of changing the original pixels falling within the first range of the F-stage value includes a process of changing the original pixels by applying a first color to the original pixels, and the process of changing the original pixels falling within the second range of the F-stage value includes a process of changing the original pixels by applying a second color to the original pixels.

[0018] Concept 8 of the present invention is a method of generating a display related to the output signal of a measuring device, a process of receiving an original image composed of a plurality of pixels, A process for obtaining the equivalent f-stop value of the pixel, A process for generating a modified image by pseudo-coloring a part of the original image based on the equivalent f-stop value, A process for displaying the modified image and comprising the above.

[0019] Concept 9 of the present invention is the display generation method of Concept 8 above, and at this time, the process of pseudo-coloring a part of the original image includes a process of replacing at least a part of the original pixels falling within the first range of the f-stop value with color pixels.

[0020] Concept 10 of the present invention is the display generation method of Concept 8 above, and at this time, the process of pseudo-coloring a part of the original image includes a process of mixing at least a part of the original pixels falling within the first range of the f-stop value with color pixels.

[0021] Concept 11 of the present invention is a waveform monitor having an image input unit and a monitor for viewing the display of measurement values, a measurement system for measuring the luminance value of the pixels of the original image received by the image input unit, a conversion unit configured to generate an f-stop value from the luminance value, a selection unit configured to generate a range of f-stop values and a modification unit configured to modify the original image with respect to the selected pixels falling within the range of the f-stop value and comprising the above.

[0022] Concept 12 of the present invention is the waveform monitor of Concept 11 above, and at this time, the modification unit is configured to replace the selected pixels with color pixels.

[0023] Concept 13 of the present invention is the waveform monitor of Concept 11 above, and at this time, the modification unit is configured to combine the selected pixels with color pixels.

[0024] Concept 14 of the present invention is the waveform monitor of Concept 11, where the selection unit is controllable by the user.

[0025] Concept 15 of the present invention is the waveform monitor of Concept 11, where the selection unit is further configured to generate a range of second F-step values.

[0026] Concept 16 of the present invention is the waveform monitor of Concept 15, where the modification unit is further configured to modify the original image with respect to the selected pixels falling within the range of the second F-step values.

[0027] Concept 17 of the present invention is the waveform monitor of Concept 11, further comprising an output signal generation structure that generates a time-versus-F-step waveform.

[0028] Concept 18 of the present invention is the waveform monitor of Concept 11, where the converter calibrated to generate the F-step value has a look-up table (LUT).

[0029] Concept 19 of the present invention is the waveform monitor of Concept 18, where the LUT is one of a plurality of pre-stored LUTs accessible by the waveform monitor.

[0030] Concept 20 of the present invention is the waveform monitor of Concept 17, where the monitor displays the time-versus-F-step waveform and a user-controllable cursor for defining the range of the F-step value. BRIEF DESCRIPTION OF THE DRAWINGS

[0031]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0032] As described above, the embodiments of the present invention can be used to evaluate live video signals from a camera from the perspective of relative aperture steps (log 2 scale: the F value changes by 1 step every time the amount of light changes by a factor of 2) in addition to traditional linear voltage displays and IRE levels. This effectively converts even the output signal of an analog camera into relative lighting and exposure values of an exposure meter (light meter) in terms of aperture steps (f-stop). Note that IRE (Institute of Radio Engineers) is a unit when expressing the pedestal level of a composite video signal of video as 0% and the full white level as 100%. 100 IRE corresponds to 714 mV in NTSC and 700 mV in PAL.

[0033] FIG. 1 is a block diagram centering on the hardware part of an example of a video waveform monitor according to the present invention. As shown in FIG. 1, a waveform monitor 20 is connected to a camera 12 to receive an input signal. The camera 12 is directed at a subject 14, and the subject 14 is illuminated by a lighting 16. The camera 12 usually has an aperture and an exposure index adjustment mechanism, which may be controlled by the operator of the camera or may be automatically adjusted by the camera 12. The output signal of the camera 12 is input to the waveform monitor 20.

[0034] The output signal of the camera 12 is first processed by an input processor 30 and then passes through a luma filter 32. The filter 32 is, for example, a low-pass filter, and the user may turn the filter 32 on / off using a user interface (UI) 50 or the like. The filtered output signal or the unfiltered output signal is supplied to a look-up table (LUT) 40. The LUT 40 is loaded via the user interface 50 using a preset table (preset table). Several preset tables may be prepared in advance, and the user may select and load one of them. For example, as one option of the LUT 40, an LUT 40A for performing gamma correction (gamma removal) or logarithmic (log) processing on the luma signal (luminance signal) may be used. Another option of the LUT 40 is to use the luma signal Y as Log 2(Y / Ymax) scale conversion is performed, and there is a LUT40B that enables real-time waveform display of the F-step number Luma signal (such waveform display is shown in FIG. 2B). At this time, Ymax is the maximum value of the Luma signal (luminance signal), that is, the white level (700 mV in PAL). Since the logarithm with a base of 2 is used, when (Y / Ymax) doubles, the F-step number value increases by 1. For example, in this example, the F-step number value -2.4 corresponds to 132.6 mV when converted to voltage, and the F-step number value -5.9 corresponds to 11.7 mV. Thus, in the display examples shown in FIGS. 2A and 2B, the cursors 1 and 2 are in the same positions, but when considering the difference (delta) between the cursors 1 and 2 in FIG. 2B in terms of voltage conversion, it can be seen that it is significantly larger compared to the case of displaying with a linear voltage scale as shown in FIG. 2A. Also, it can be seen that the vicinity of the black level (0 voltage) can be displayed in detail compared to a linear voltage scale. To reduce the size of the LUT, both multiplication and logarithmic scaling operation processes are accurately performed in advance, and all words of the LUT are converted into a single data set, so a large word size is not required to represent a wide dynamic range of linear light quantity values.

[0035] As described above, there may be a plurality of LUTs stored in the waveform monitor 20. The user can control which of the plurality of stored LUTs is loaded as the active LUT40 using the user interface 50. For example, various LUTs may be stored in the waveform monitor 20 in advance, so that the user can select the active LUT40 based on the gamma and black level of the camera.

[0036] The display monitor (display device) 60 on the waveform monitor 20 displays the output signal to the user. Monitor 60 can be used to display a traditional time-versus-voltage waveform as shown in FIG. 2A, while it can also be used to show a new time-versus-F-step waveform as shown in FIG. 2B. In this new time-versus-F-step waveform display, similar to the traditional time-versus-voltage waveform display, the user can utilize an adjustable cursor, but the scale of the output signal is different in that it is "F-step" instead of voltage. As shown in FIGS. 2A and 2B, the horizontal time axis is the same for both displays, and conventional scales such as, for example, 1 line, 2 lines, field sweep, etc. can be selected.

[0037] In addition, the cursor range (window) selection block 70 receives input from the user through the user interface 50 and reads a plurality of variable cursor values set by the user. These variable cursor values are utilized as binary gate signals to modify the monochrome output signal of the waveform monitor 20. Specifically, the input processor 30 receives the input signal from the camera and performs the necessary processing, and the color cursor mixer 80 is coupled to the input processor 30 and receives the processed signal from the input processor 30 as the processed input signal. Note that the color cursor mixer 80 may also be configured to receive the processed input signal from another component of the waveform monitor 20. The color cursor mixer 80 is also coupled to the cursor range selection block 70. The binary gate signal from the cursor range selection block 70 is utilized to determine which region of the original monochrome output signal to color, i.e., the region to be highlighted in the output signal. In this way, the color cursor mixer 80 functions as an image modification unit.

[0038] For example, when comparing the output signals of FIGS. 3 and 6, FIG. 3 is the original monochrome output signal, while the output signal of FIG. 6 has a portion highlighted in color to indicate the image area falling within the F-step range selected by the user on the original monochrome image. The captured image shown in FIG. 3 is only luma (brightness), i.e., a monochrome image, while in FIG. 6, as will be described in detail later, a cursor range (for example, red and blue) by two colors is added. Note that in FIG. 6, since the drawing is in grayscale, the red area is shown as a horizontal stripe pattern and the blue area is shown as a vertical stripe pattern (such as the lower part of the chin of the person).

[0039] Referring to FIG. 1 again, to generate the output signal of FIG. 6, in the detection circuit within the cursor range selection block 70, the log 2 (Y / Ymax) signal is compared with two cursor ranges controllable by the user. The user can control the position and size of the cursor range. Specifically, in one embodiment, the user may control the center value of the F-step value and the size of the cursor range. In another embodiment, the user may define the cursor range using two cursors. In the system, a binary gate signal is generated using this range. When the F-step value (luminance value) of a pixel in the luma output signal falls within the specified cursor range, that pixel is shown as a colored pixel in the display, thereby providing the user with information on which pixels in the output signal are within the specified F-step range. The remaining pixels, i.e., the pixels having a luminance level outside the specified cursor range in the original pixels, are shown without change in the display.

[0040] The cursor range may be adjustable, for example, in 1 / 4-step increments. Also, for example, the cursor range may be preset to a range of plus or minus 1 / 4-step from the center value of the F-stop number set by the user. The center value of the F-stop number is adjustable across the entire range of the signal obtained from the LUT 40. In this method, the user can adjust one cursor to highlight a specific area of the image, and thereby, in addition to the specific area on the image having the luminance value of that F-stop number from the value of that cursor on the F-stop number waveform shown in FIG. 2B or FIG. 5, since there is a width of plus or minus 1 / 4-step, other areas having luminance values of approximately the same F-stop number can be identified.

[0041] The color cursor mixer 80 generates, for example, coloring signals for red and blue pixels and synthesizes them with the original image to generate a modified image as shown in FIG. 6. In one embodiment, the color cursor mixer 80 simply replaces the original pixels falling within the set F-stop number range with single-color red or blue pixels. In another embodiment, the color cursor mixer 80 may generate a color-mixed output signal by adding a color tone to the underlying luminance data. Also, in these examples, a monochrome-based image is shown, but embodiments of the present invention are not limited to those based on luma (luminance), and may be implemented independently, for example, in each channel of red, green, and blue.

[0042] In the embodiment shown in FIG. 6, there are two-color cursor ranges, but the waveform monitor 20 can also generate a greater number of cursor ranges or a fewer number of cursor ranges.

[0043] FIG. 4 shows the traditional voltage waveform (VmV_wfm n ) of the image in FIG. 3. FIG. 5 similarly shows the output signal for the image in FIG. 3, but in FIG. 5, it shows the F-stop number waveform (Fstop_wfm n ) for the image in FIG. 3 according to an embodiment of the present invention, having two cursors, FS low (meaning low F-stop number) and FS high (meaning high F-stop number).

[0044] By using the embodiments of the present invention, in conjunction with the lighting of the shooting scene, the adjustment of the camera's gain / aperture / shutter speed can be easily performed in familiar F-stop units, and the camera can be effectively used as an exposure meter. For example, by using the embodiments of the present invention, as shown in FIG. 5, the user can view a waveform weighted by the F-stop on a display showing graduations of a linearized scale in F-stop units. Further, according to the embodiments of the present invention, by enabling the user to control the differential measurement value of the delta F-stop of the content elements of the image / scene by using the F-stop waveform and the pseudo-color region on the monochrome image display, the user can measure the hot spots and lighting non-uniformity of the scene in F-stop units using two cursors. Furthermore, according to the embodiments of the present invention, the characteristics of the camera (camera matching) can be adjusted by performing accurate black level adjustment and high-resolution black balance adjustment (adjusting the levels of the three primary RGB colors to accurately reproduce black), and the noise of the camera can be displayed in high resolution. Furthermore, the embodiments of the present invention can be used as a tool for evaluating the dynamic range of the content of a video scene, and thereby can be used to optimize the dynamic range from the perspective of artistic quality (video expression).

[0045] The waveform monitor 20 and any of its components may be implemented in firmware in a circuit or FPGA designed for a specific purpose such as an ASIC, or may be implemented as one or more software processes executed on one or more processors. In another embodiment, the waveform monitor 20 may include, for example, a combination of components or processes executed on firmware, ASIC, FPGA, and software.

[0046] Specific embodiments of the present invention have been described by way of examples and for the purpose of explaining the examples, but it will be apparent that various modifications are possible without departing from the spirit and scope of the present invention.

Description of Reference Numerals

[0047] 12 Camera 14 Object to be written 16 Writing 20 Waveform monitor 32 Luma filter (brightness filter) 40 F-step lookup table (conversion unit) 50 User interface 60 Display device (monitor) 70 Cursor range selection block (selection unit) 80 Color cursor mixer (image modification unit)

Claims

1. A waveform monitor having an image input unit and a monitor for viewing a display of a measurement value, a measurement system for measuring pixel luminance values ​​from an original image received by the image input unit; The measured luminance value is subjected to gamma removal or logarithmic processing, and the luminance value subjected to the gamma removal or logarithmic processing is converted into an equivalent F-stage number Log by using a lookup table. 2 a converter configured to generate an equivalent F-step value from the measured luminance values ​​by converting the measured luminance values ​​into a (Y / Ymax) scale, where Y is the luminance value of the measured luminance values ​​and Ymax is the maximum luminance value of the measured luminance values; a selector configured to generate a range of first F-stage values; a color mixer configured to produce colored pixels for selected pixels of the original image that fall within the first F-stage range and to pass monochrome pixels for unselected pixels of the original image as they are; an output signal generating structure configured to display an image modified from the original image by the color mixer; A waveform monitor comprising:

2. 2. The waveform monitor of claim 1, wherein the color mixer is configured to replace the selected pixel with a color pixel and / or combine the selected pixel with a color pixel.

3. 3. The waveform monitor of claim 1 or 2, wherein the selection is user controllable.

4. 4. The waveform monitor of claim 1, wherein the selector is further configured to generate a second range of F-stage values.

5. 5. The waveform monitor of claim 4, wherein said color mixer is further configured to modify said original image for selected pixels that fall within said second F-stage value range.

6. 2. The waveform monitor of claim 1, wherein the lookup table is one of a plurality of pre-stored lookup tables accessible by the waveform monitor.

7. 1. A method for generating an indication of an output from a measurement device, comprising: receiving an original image comprising a plurality of pixels; extracting luminance information from the plurality of pixels; The extracted luminance information is subjected to gamma removal or logarithmic processing, and the luminance information subjected to the gamma removal or logarithmic processing is converted into an equivalent F-stage Log by using a lookup table. 2 converting the luminance information of the plurality of pixels into an equivalent F-step value by converting the luminance information of the plurality of pixels into an equivalent F-step value on a (Y / Ymax) scale, where Y is a luminance value of the extracted luminance information and Ymax is a maximum luminance value of the extracted luminance information; selecting said equivalent F-step value falling within a first range of F-step values; modifying the original image by modifying pixels of the original image that fall within the first range of F-stage values ​​and leaving remaining pixels outside the first range of F-stage values ​​monochrome to generate a modified image; Displaying the above modified image 13. A method for generating an output display comprising:

8. 8. A method for generating an output display as recited in claim 7, wherein the magnitude of said first range of said F-step values ​​is user adjustable.

9. 9. A method as claimed in claim 7 or 8, wherein the centre value of said first range of said F-step values ​​is user adjustable.

10. 8. The method of claim 7, wherein modifying the original image to generate the modified image by modifying pixels of the original image includes replacing at least a portion of pixels of the original image that fall within the first range of F-stage values ​​with color pixels.

11. 8. The method of claim 7, wherein modifying the original image to generate the modified image by modifying pixels of the original image includes combining luminance values ​​of at least some of the pixels of the original image that fall within the first range of F-stage values ​​with color values.

12. selecting an equivalent F-step value that falls within a second range of F-step values; and modifying the original image by modifying pixels of the original image that fall within the second range of F-stage values ​​to generate a modified image; Further comprising:

8. The method of claim 7, wherein modifying pixels of the original image that fall within the second range of F-stage values ​​is different from modifying pixels of the original image that fall within the first range of F-stage values.

13. 13. The method of claim 12, wherein modifying pixels of the original image that fall within the first range of F-stage values ​​includes modifying the pixels of the original image by applying a first color, and wherein modifying pixels of the original image that fall within the second range of F-stage values ​​includes applying a second color.

Images