Signal processing device and signal processing device control method as well as program

Abstract

To provide a signal processing device which can easily analyze a virtual region and a real region separately and is suitable for virtual production, and a signal processing device control method.SOLUTION: The signal processing device comprises: first obtaining means that obtains image data that can be obtained when a photographing device photographs a subject and a background video displayed on a background display; first generating means that generates a first index indicating at least one of characteristics including luminance, color and spatial frequency of a background video region; second generating means that generates a second index indicating the same characteristics as that of the first generating means of characteristics including luminance, color and spatial frequency of a subject region; and synthesizing means that synthesizes the first index, the second index and the image data with one another to output synthesized image data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a signal processing device, a control method for the signal processing device, and a program. [Background technology]

[0002] Virtual production of in-camera VFX using LED walls is becoming more common. An LED wall is a display device with LEDs arranged in a grid pattern. In in-camera VFX, a previously captured image is displayed on the LED wall, and performers and art sets are placed between the LED wall and the camera, and the LED wall, performers, and art sets are photographed. Of the images captured by in-camera VFX, the partial area in which the performers and art sets are photographed is referred to here as the real area (subject area), and the partial area in which the LED wall is photographed is referred to here as the virtual area (background image area).

[0003] In virtual production, there is a demand for the ability to analyze the virtual and real areas separately using equipment such as waveform monitors and vectorscopes. The virtual area is an image taken by the first camera, which is displayed on an LED wall and then photographed again by the second camera, while the real area is an image taken directly by the second camera. Due to differences in the shooting techniques, the image quality of the virtual and real areas may differ, and this must be confirmed. The images referred to here include both video and still images.

[0004] For example, Patent Document 1 discloses a technique for displaying a vectorscope for a partial region of an image. As a method for specifying the partial region, a method in which the user specifies the partial region using a touch panel is disclosed. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2017-16260 A Summary of the Invention [Problem to be solved by the invention]

[0006] Although the method of Patent Document 1 can display a vector scope of a partial area, the user needs to specify the area in order to analyze the virtual area and the real area separately. Also, the user needs to specify the area every time the area moves in the video.

[0007] Therefore, the present invention provides a signal processing device and a signal processing method that can easily analyze a virtual area and a real area separately and are suitable for virtual production. [Means for solving the problem]

[0008] The technical features of the present invention include a first acquisition step of acquiring image data obtained by capturing an image of a subject and a background image displayed on a background display using an imaging device; an extraction step of extracting information on the subject area and the background image area in the image data; a first generation step of generating a first index indicating at least one of the luminance, color, and spatial frequency characteristics of the subject area; a second generation step of generating a second index having the same characteristics as those of the luminance, color, and spatial frequency characteristics of the background image area in the first generation step; and a synthesis step of synthesizing the first index, the second index, and the image data to output synthetic image data. Effect of the Invention

[0009] According to the present invention, it is possible to provide a signal processing device and a signal processing method that are suitable for virtual production, by simply and easily analyzing a virtual area and a real area separately. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a block diagram showing an example of the configuration of a signal processing device according to the first, fifth, seventh and eighth embodiments. [Diagram 2]1 is a block diagram showing an example of the configuration of an imaging device according to the first, second, third, fourth, fifth, seventh and eighth embodiments. [Diagram 3] FIG. 1 is a block diagram showing an example of device connections according to the first, fifth, seventh and eighth embodiments. [Figure 4] FIG. 13 is a diagram showing an outline of in-camera VFX shooting from a bird's-eye view according to the first, second, third, fourth, fifth, seventh and eighth embodiments. [Diagram 5] FIG. 11 is a diagram showing an example of captured video data according to the first, third, fourth, fifth and eighth embodiments. [Figure 6] FIG. 11 is a diagram showing an example of display of composite moving image data according to the first and fifth embodiments. [Figure 7] FIG. 11 is a block diagram showing an example of the configuration of a signal processing device according to a second embodiment. [Figure 8] FIG. 11 is a block diagram showing an example of the configuration of a display device according to a second embodiment. [Figure 9] FIG. 11 is a block diagram showing an example of device connections according to a second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of captured video data according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of display of composite moving image data according to the second embodiment. [Figure 12] FIG. 11 is a diagram showing an example of drawing on a first waveform monitor according to the second embodiment. [Figure 13] FIG. 11 is a diagram showing an example of drawing a first chromaticity diagram according to a second embodiment. [Figure 14] FIG. 11 is a block diagram showing an example of the configuration of a signal processing device according to a third embodiment. [Figure 15] FIG. 11 is a block diagram showing an example of device connections according to a third embodiment. [Figure 16] FIG. 13 is a diagram showing an example of display of composite moving image data according to the third embodiment. [Figure 17] FIG. 13 is a block diagram showing an example of the configuration of a signal processing device according to a fourth embodiment. [Figure 18] FIG. 13 is a block diagram showing an example of device connections according to a fourth embodiment. [Figure 19] FIG. 13 is a diagram showing an example of display of composite moving image data according to the fourth embodiment. [Figure 20] FIG. 13 is a diagram showing an example of drawing on a first waveform monitor according to the fourth embodiment. [Figure 21] 13 is a flowchart showing a control flow according to the fifth embodiment. [Figure 22] FIG. 13 is a diagram showing an example of display of composite moving image data according to the fifth embodiment. [Figure 23] FIG. 13 is a block diagram showing an example of the configuration of a signal processing device according to a sixth embodiment. [Figure 24] FIG. 23 is a diagram showing an example of first input moving image data and second input moving image data according to the sixth embodiment. [Diagram 25] FIG. 23 is a diagram showing an example of video data according to the sixth embodiment. [Figure 26] FIG. 23 is a diagram showing an example of display of composite moving image data according to the sixth embodiment. [Figure 27] FIG. 23 is a diagram showing an example of display of composite moving image data according to the seventh embodiment. [Figure 28] FIG. 23 is a diagram showing an example of display of composite moving image data according to the seventh embodiment. [Figure 29] FIG. 23 is a diagram showing an example of display of composite moving image data according to the eighth embodiment. [Diagram 30] FIG. 23 is a diagram showing an example of a distance graph according to the eighth embodiment. [Diagram 31] FIG. 23 is a diagram showing an example of a distance graph according to the eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0012] (First embodiment) ●Signal processing device configuration Fig. 1 is a block diagram showing an example of the configuration of a signal processing device according to this embodiment. The signal processing device 100 in Fig. 1 includes a receiving unit 101, a detecting unit 102, a control unit 103, a memory unit 104, a synthesizing unit 105, a display unit 106, a generating unit 107, and a generating unit 108.

[0013] The receiving unit 101 acquires captured video data 109, and outputs the video data to the detecting unit 102, the synthesizing unit 105, the generating unit 107, and the generating unit 108. In this embodiment, the receiving unit 101 acquires image data for each frame of a video from an external device. The receiving unit 101 then outputs the acquired image data to each unit in the subsequent stage. The receiving unit 101 is an input terminal that complies with standards such as SDI and HDMI (registered trademark). The external device is an imaging device, a playback device, or the like. In this embodiment, an example of connection to an imaging device 200 will be described, and details will be described later.

[0014] The detection unit 102 acquires the video data including the region information output from the reception unit 101, and outputs the acquired region information to the control unit 103, the generation unit 107, and the generation unit 108. The region information will be described later.

[0015] The control unit 103 controls the processing of each unit of the signal processing device 100. The control unit 103 is connected to each unit via a control bus, but the control bus is not shown in the figure because it is complicated. For example, the control unit 103 is an arithmetic processing circuit that executes a program stored in the memory unit 104 to control the processing of each block of the signal processing device 100. In this embodiment, the control unit 103 controls the synthesis unit 105 based on the region information output from the detection unit 102.

[0016] The memory unit 104 stores programs, parameters, etc. The programs and parameters stored in the memory unit 104 are read and written by each block of the signal processing device 100.

[0017] The synthesis unit 105 synthesizes the moving image data with the first waveform monitor, which is a first indicator output by the generation unit 107, and the second waveform monitor, which is a second indicator output by the generation unit 108, based on the control content output from the control unit 105, to generate synthetic image data. The synthetic image data is repeatedly generated to generate synthetic moving image data. Then, the synthesis unit 105 outputs the synthetic moving image data to the display unit 106.

[0018] The display unit 106 displays on a display surface a moving image based on the composite moving image data output from the composition unit 105. The display unit 106 is, for example, a display panel configured with a liquid crystal panel and a backlight unit, an organic EL display panel, or the like.

[0019] The generating unit 107 acquires the video data output from the receiving unit 101 and also acquires the area information output from the detecting unit 102, and generates a waveform monitor (first waveform monitor) related to pixels designated as a virtual area by the area information in the video data. More specifically, one frame's worth of data is extracted from the video data, and pixels designated as a virtual area are extracted from that data, and the analysis results are generated as the first waveform monitor.

[0020] The generating unit 108 acquires the video data output from the receiving unit 101 and also acquires the area information output from the detecting unit 102, and generates a waveform monitor (second waveform monitor) related to pixels designated as real areas by the area information in the video data. More specifically, one frame's worth of data is extracted from the video data, and pixels designated as real areas are extracted from that data, and the results of analyzing these are generated as the second waveform monitor.

[0021] The operation of each unit has been described, but the procedure for integrating the operations and displaying the first waveform monitor and the second waveform monitor on the display unit 106 will now be described. When the receiving unit 101 receives captured video data 109, the detecting unit 102 extracts area information from the video data and outputs it to the control unit 103 and the generating units 107 and 108. The generating units 107 and 108 refer to the area information and generate the first waveform monitor and the second waveform monitor. The combining unit 105 combines the first waveform monitor and the second waveform monitor into video data and outputs it to the display unit 106 as combined video data. The display unit 106 displays the combined video data including the first waveform monitor and the second waveform monitor.

[0022] Imaging device configuration 2 is a block diagram showing an example of the configuration of an image capturing device that generates captured video data acquired by a signal processing device according to this embodiment. The image capturing device 200 in FIG. 2 includes an optical unit 201, an image capturing unit 202, a region information superimposing unit 203, a transmitting unit 204, and a distance measuring unit 205.

[0023] The optical unit 201 includes a lens, a focus motor, and the like, and forms an image of an object to be photographed on the imaging unit 202. The imaging unit 202 includes an image sensor, and the like, and generates video data from the image formed by the optical unit 201, and outputs the video data to the area information superimposing unit 203. The area information superimposing unit 203 superimposes area information on the video data, and outputs the video data to the transmitting unit 204. The area information will be described later. The transmitting unit 204 transmits the video data received from the area information superimposing unit 203 as the captured video data 109. The transmitting unit 204 is an output terminal that complies with standards such as SDI and HDMI. The distance measuring unit 205 measures the distance between the imaging device and the object to be photographed for each pixel on the image sensor. The distance measured by the distance measuring unit 205 is used to generate area information, and the details will be described later. There is known a technology for measuring distance by arranging pixels that detect phase differences on an image sensor, and in this case, the imaging unit 202 and the distance measuring unit 205 are integrated into one unit.

[0024] ● Device connection example 3 is a block diagram showing an example of connections between an image generating device, a display device, an imaging device, and a signal processing device. The image generating device 400 generates moving image data by storing pre-captured moving image data inside and playing it back. The image generating device 400 outputs the generated moving image data to the display device 300 as generated moving image data 301. The display device 300 is an LED wall, and is a background display that displays the generated moving image data 301 (background video data). The imaging device 200 outputs the captured moving image data to the signal processing device 100 as captured moving image data 109.

[0025] Overview of the shoot FIG. 4 is a diagram showing an overview of in-camera VFX shooting from a bird's-eye view. Actor 503 performs between display device 300 and imaging device 200. Imaging device 200 captures display device 300 and actor 503. The dotted line indicates the angle of view of optical unit 201. Note that while FIG. 4 shows a case where there is one actor who is the subject to be captured, there may be multiple actors, or an art set in addition to the actors. A background image is displayed on display device 300 and captured by imaging device 200.

[0026] - Area information to distinguish between real and virtual areas Here, the area information will be described. The area information is information for identifying whether each pixel is a real area or a virtual area. The area information superimposing unit 203 assigns area information to pixels whose distance measured by the distance measuring unit 205 is approximately equal to the distance between the imaging device 200 and the display device 300 as a virtual area, and other pixels as a real area. The distance between the imaging device 200 and the display device 300 is measured by the users of the imaging device 200 and the display device 300 and set in the imaging device 200. The area information is superimposed on the video data as metadata. Examples of the metadata include ANC data standardized in SDI and InfoFrame standardized in HDMI. In the ANC data, any binary data can be superimposed on the horizontal blanking period. Although a specific method for superimposing the area information is not standardized, the area information can be superimposed by using the ANC data as follows. A 1-bit value 1 indicates that one pixel is a virtual area, and a 1-bit value 0 indicates that one pixel is a real area. Data summarizing the values ​​(1 or 0) of the number of pixels for one line is generated and superimposed as ANC data during the horizontal blanking period of the corresponding line.

[0027] ●An example of video data FIG. 5 is a diagram showing an example of shot video data 109. Wall 501 and window frame 502 are an art set. A performer 503 is positioned in front of wall 501 of the art set. Outside view 511 and standing tree 512 are positioned inside window frame 502. Outside view 511 and standing tree 512 are photographed in advance and displayed on display device 300. In this state, when wall 501, window frame 502, performer 503, and display device 300 are photographed, the shot video data shown in FIG. 5 is obtained. Wall 501, window frame 502, and performer 503 are real areas. Outside view 511 and standing tree 512 are virtual areas.

[0028] An example of composite video data displayed on the display unit FIG. 6 is a diagram showing an example of the composite video data displayed by the display unit 106. The waveform monitor 601 is a first waveform monitor generated by the generation unit 107, and visualizes the characteristics by analyzing the virtual area of ​​the captured video data shown in FIG. 5. The waveform monitor 602 is a second waveform monitor generated by the generation unit 108, and visualizes the characteristics by analyzing the real area of ​​the captured video data shown in FIG. 5. The same reference numerals as in FIG. 5 are used in FIG. 6 as well. The captured video data shown in FIG. 5 is composited with the waveform monitors 601 and 602 to produce the composite video data shown in FIG. 6. The composite video data shown in FIG. 6 is displayed by the display unit 106.

[0029] Effects of this embodiment According to this embodiment, when analyzing the virtual area and the real area separately, the user does not need to specify the area. By referring to the analysis results of the first waveform monitor and the second waveform monitor, it becomes easy to check the brightness distribution of the virtual area and the real area and adjust the exposure of the imaging device and the brightness of the LED wall.

[0030] Variations In the present embodiment, an example in which separate waveform monitors are displayed for the virtual region and the real region has been described, but instead of the waveform monitor, an image analysis means including a vectorscope and a chromaticity diagram can be used. In addition, in the present embodiment, an example in which the synthesis unit 105 synthesizes the first and second waveform monitors with the video data and outputs the result to the display unit 106 has been described. The synthesis unit 105 may output the first and second waveform monitors to the display unit 106, and the display unit 106 may display the first and second waveform monitors. In this case, the display unit 106 does not display the video data. In addition, in the present embodiment, an example in which the signal processing device 100 receives video data has been described, but still image data may be received. In the case of a video, the receiving unit 101 acquires image data for each frame of the video, but in the case of a still image, the receiving unit 101 acquires new image data each time the still image data is changed.

[0031] Second embodiment The second embodiment of the present invention will be described below. However, since this embodiment is a modified version of the first embodiment, the description will focus on the differences from the first embodiment, and the same content as the first embodiment will be omitted as appropriate.

[0032] ●Signal processing device configuration FIG. 7 is a block diagram showing an example of the configuration of a signal processing device according to this embodiment. The signal processing device 700 in FIG. 7 includes a receiving unit 101, a detecting unit 102, a control unit 103, a memory unit 104, a synthesizing unit 105, a display unit 106, a generating unit 108, a reference value acquiring unit 701, and a generating unit 702. The reference value acquiring unit 701 acquires a reference value 703 from an external display device 800. The reference value 703 is a value that is a reference for the brightness and color tone of the display device 800. The display device 800 will be described later. The generating unit 702 acquires the moving image data output from the receiving unit 101 and also acquires the area information output from the detecting unit 102, and generates a waveform monitor (first waveform monitor) related to pixels designated as a virtual area by the area information in the moving image data. Furthermore, the generating unit 702 draws the reference value of the display device 800 acquired by the reference value acquiring unit 701 on the first waveform monitor. The receiving unit 101, the detecting unit 102, the control unit 103, the memory unit 104, the combining unit 105, the display unit 106, and the generating unit 108 are the same as those in the first embodiment.

[0033] Display device configuration FIG. 8 is a block diagram showing an example of the configuration of the display device according to this embodiment. The display device 800 in FIG. 8 includes a receiving unit 801, a display unit 802, and a reference value output unit 803. The receiving unit 801 acquires the generated video data 301 and outputs the video data to the display unit 802. The receiving unit 801 is an input terminal conforming to a standard such as SDI or HDMI (registered trademark). The display unit 802 displays a video based on the video data output from the receiving unit 801 on a display surface. The display unit 802 arranges LEDs in a matrix to form an LED wall. The reference value output unit 803 outputs a reference value 703 that is a reference for the brightness and color of the display device 800. The reference value 703 may be, for example, a maximum brightness of the display device 800, a brightness of the display device 800 corresponding to a reflectance of 18%, a brightness of the display device 800 corresponding to a reflectance of 100%, brightness 200nits, brightness 203nit At least one of In another example, the reference value 703 is a reference value of color, which is the chromaticity of the achromatic color of the display device 800, and the chromaticity of red, green, and blue that constitute the color gamut of the display device 800.

[0034] ● Device connection example 9 is a block diagram showing an example of connections between an image generating device, a display device, an imaging device, and a signal processing device. The image generating device 400 outputs generated moving image data as generated moving image data 301 to the display device 800. The imaging device 200 outputs captured moving image data as captured moving image data 109 to the signal processing device 700. Furthermore, the display device 800 outputs a reference value 703 to the signal processing device 700. For example, a LAN (Local Area Network) can be used for communication between the display device 800 and the signal processing device 700.

[0035] ●Shooting video data and composite video data The following describes a case where the reference value 703 is the brightness reference. Fig. 10 is a diagram showing an example of captured video data 109. The wall 501, window frame 502, and performer 503 are the same as in the first embodiment. The display device 800 displays a uniform achromatic color 1001. When the wall 501, window frame 502, performer 503, and display device 800 are captured, the captured video data shown in Fig. 10 is obtained. The wall 501, window frame 502, and performer 503 are real areas. The achromatic color 1001 is a virtual area.

[0036] Fig. 11 is a diagram showing an example of the composite video data displayed by the display unit 106. The waveform monitor 1101 is a first waveform monitor generated by the generation unit 702, and visualizes the features by analyzing the virtual area of ​​the captured video data shown in Fig. 10. The waveform monitor 602 is a second waveform monitor generated by the generation unit 108, and visualizes the features by analyzing the real area of ​​the captured video data shown in Fig. 10. Items that are the same as those in Fig. 10 are the same as those in Fig. 10. The captured video data shown in Fig. 10 is composited with the waveform monitor 1101 and the waveform monitor 602 to produce the composite video data shown in Fig. 11.

[0037] Example of the first waveform monitor FIG. 12 is an example of a display on the first waveform monitor. The vertical axis of the first waveform monitor is luminance, and its range is 0 nit to 10000 nit. Display device 800 When the maximum luminance is acquired and the value is 1000 nit, the generation unit 702 draws the drawing 1201 at the position of 1000 nit on the vertical axis. The brightness value of is rendered as drawing 1202. In the example shown in FIG. The brightness value of However, since the image is drawn at a position higher than 1000 nit represented by the drawing 1201, exposure It can be seen that the setting is higher than the actual setting. Note that, although the example in which the reference value is the maximum brightness of 1000 nit has been described in the present embodiment, the present invention is not limited to this.

[0038] The following describes a case where the reference value 703 is a reference for color tone. The generating unit 702 acquires the moving image data output from the receiving unit 101 and also acquires the area information output from the detecting unit 102, and generates a chromaticity diagram (first chromaticity diagram) for pixels designated as a virtual area by the area information in the moving image data. Furthermore, the generating unit 702 draws the reference value 703 of the display device 800 acquired by the reference value acquiring unit 701 on the first chromaticity diagram.

[0039] Example of drawing the first chromaticity diagram 13 is a drawing example of the first chromaticity diagram. The horizontal axis of the first chromaticity diagram is x chromaticity, and the vertical axis is y chromaticity. Display device 800 When the chromaticity of the achromatic color is acquired and the value is (x=0.3127, y=0.3290), the generation unit 702 draws drawing 1302 at the corresponding position on the chromaticity diagram. The achromatic color 1001 of the captured video data shown in FIG. 10 is drawn as drawing 1303. In the example shown in FIG. 13, the achromatic color 1001 is in a position where x is larger and y is smaller (a position with a relatively stronger reddish hue) than the (x=0.3127, y=0.3290) represented by drawing 1302. From this, it can be understood that the settings of the imaging device 200 are set to be more reddish than they should be. It should be noted that the reference values ​​(x=0.3127, y=0.3290) in this embodiment are an example in which the achromatic color of D65 is used, but reference values ​​based on another standard light source may be used.

[0040] Effects of this embodiment According to this embodiment, the following is true regarding brightness and color: Reference value of display device 800 For example, in the example shown in FIG. 12, it can be seen that the captured video data is brighter than the reference value. This is because the settings of the imaging device are Reference value of display device 800 than exposure It's over (In other words, the subject that should have the reference white luminance after in-camera FVX is not captured at the reference white luminance.) In the example shown in FIG. 13, it can be seen that the captured video data has a redder hue than the reference value, which suggests that the settings of the imaging device are Reference value of display device 800 This suggests that the setting has a stronger reddish hue than the original.

[0041] (Third embodiment) The third embodiment of the present invention will be described below. However, since this embodiment is a modified version of the first embodiment, the description will focus on the differences from the first embodiment, and the same content as the first embodiment will be omitted as appropriate.

[0042] ●Signal processing device configuration Fig. 14 is a block diagram showing an example of the configuration of a signal processing device according to this embodiment. A signal processing device 1400 in Fig. 14 includes a receiving unit 101, a detecting unit 102, a control unit 103, a memory unit 104, a synthesizing unit 105, a display unit 106, a generating unit 107, a generating unit 108, a receiving unit 1401, a converting unit 1402, and a generating unit 1403. Here, the receiving unit 101, the detecting unit 102, the control unit 103, the memory unit 104, the synthesizing unit 105, the display unit 106, the generating unit 107, and the generating unit 108 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0043] The receiving unit 1401 acquires the generated video data 301 from the external image generating device 400 , and outputs the video data to the converting unit 1402 .

[0044] The conversion unit 1402 converts the video data and outputs it to the generation unit 1403 . The conversion unit 1402 performs calculations on the generated video data 301 to convert the display device 300 displaying the generated video data 301 into video data captured from the position of the imaging device 200, with the optical axis of the imaging device 200 as the viewpoint and the imaging range of the imaging device 200 as the field of view. The four end points of the imaging range on the imaging surface of the imaging device 200 are (V0, V1, V2, V3), and the four points on the display surface of the display device 300 corresponding to (V0, V1, V2, V3) are (D0, D1, D2, D3). The conversion process of the conversion unit 1402 can be realized by projective transformation disclosed in Patent Document JP-A-2011-48415. That is, the four points (D0, D1, D2, D3) of the display device 300 are projectively transformed to the four points (V0, V1, V2, V3) of the imaging device 200. The image generating device 400 manages (V0, V1, V2, V3) and (D0, D1, D2, D3) required for the calculation process, and outputs the data by superimposing them on the generated video data 301. The conversion unit 1402 refers to the data via the receiving unit 1401.

[0045] The generating unit 1403 acquires the moving image data output from the converting unit 1402, and also acquires the area information output from the detecting unit 102. Then, it generates a waveform monitor (third waveform monitor) that is an index related to pixels designated as a virtual area by the area information in the moving image data. Note that, due to the conversion by the converting unit 1402, the moving image data and area information output from the converting unit 1402 can be handled in a coordinate system on the imaging plane of the imaging device 200.

[0046] ● Device connection example 15 is a block diagram showing an example of connections between an image generating device, a display device, an imaging device, and a signal processing device. The image generating device 400 outputs generated moving image data as generated moving image data 301 to the display device 300 and the signal processing device 1400. The imaging device 200 outputs captured moving image data as captured moving image data 109 to the signal processing device 1400. 1400 Output to.

[0047] An example of composite video data displayed on the display unit FIG. 16 is a diagram showing an example of the composite video data displayed by the display unit 106. The waveform monitor 601 is a first waveform monitor generated by the generation unit 107, and visualizes the features by analyzing the virtual area of ​​the captured video data shown in FIG. 5. The waveform monitor 602 is a second waveform monitor generated by the generation unit 108, and visualizes the features by analyzing the real area of ​​the captured video data shown in FIG. 5. The waveform monitor 1601 is a third waveform monitor, which is a third index generated by the generation unit 1403, and visualizes the features by extracting a virtual area from the video data output by the conversion unit 1402 based on the area information output by the detection unit 102 and analyzing the area. The same reference numerals as in FIG. 5 are used in FIG. 16 as well. The captured video data shown in FIG. 5 is composited with the waveform monitor 601, the waveform monitor 602, and the waveform monitor 1601 to produce the composite video data shown in FIG. 16.

[0048] Effects of this embodiment According to this embodiment, the difference in brightness between the generated video data and the captured video data can be intuitively known. For example, if the analysis result of the first waveform monitor is brighter than that of the third waveform monitor, it can be known that the captured video data is brighter than the generated video data. This is because the settings of the imaging device are originally than exposure Conversely, if the analysis result of the first waveform monitor is darker than that of the third waveform monitor, it means that the captured video data is darker than the generated video data. This suggests that the settings of the imaging device are too dark. originally than exposure This suggests that it is undervalued.

[0049] (Fourth embodiment) The fourth embodiment of the present invention will be described below. However, since this embodiment is a modified version of the first embodiment, the description will focus on the differences from the first embodiment, and the same content as the first embodiment will be omitted as appropriate.

[0050] ●Signal processing device configuration Fig. 17 is a block diagram showing an example of the configuration of a signal processing device according to this embodiment. A signal processing device 1700 in Fig. 17 includes a receiving unit 101, a detecting unit 102, a control unit 103, a memory unit 104, a synthesizing unit 105, a display unit 106, a generating unit 107, a generating unit 108, a receiving unit 1401, a converting unit 1402, an average brightness calculating unit 1701, and an average brightness calculating unit 1702. The receiving unit 101, the detecting unit 102, the control unit 103, the memory unit 104, the synthesizing unit 105, the display unit 106, the generating unit 107, and the generating unit 108 are the same as those in the first embodiment. The receiving unit 1401 and the converting unit 1402 are the same as those in the third embodiment.

[0051] The average brightness calculation unit 1701 calculates the average brightness of the virtual region as an average value from the video data output by the receiving unit 101 based on the region information output by the detection unit 102, and draws it on the first waveform monitor.

[0052] The average brightness calculation unit 1702 calculates the average brightness of the virtual region from the video data output by the receiving unit 1401 based on the region information output by the detection unit 102, and draws it on the first waveform monitor.

[0053] ● Device connection example 18 is a block diagram showing an example of connections between an image generating device, a display device, an imaging device, and a signal processing device. The image generating device 400 outputs generated moving image data as generated moving image data 301 to the display device 300 and the signal processing device 1700. The imaging device 200 outputs captured moving image data as captured moving image data 109 to the signal processing device 1700.

[0054] An example of composite video data displayed on the display unit Fig. 19 is a diagram showing an example of the composite video data displayed by the display unit 106. Waveform monitor 1901 is a first waveform monitor, which visualizes the characteristics by analyzing the virtual area of ​​the captured video data shown in Fig. 5. Waveform monitor 602 is a second waveform monitor, which visualizes the characteristics by analyzing the real area of ​​the captured video data shown in Fig. 5. Items that are the same as those in Fig. 5 are indicated with the same reference numerals as in Fig. 5. Waveform monitor 1901 and waveform monitor 602 are composited with the captured video data shown in Fig. 5 to produce the composite video data shown in Fig. 19.

[0055] Example of the first waveform monitor FIG. 20 is an example of rendering on the first waveform monitor. Average brightness 2001 is the average brightness of the virtual area of ​​the captured video data 109 calculated by the average brightness calculation unit 1701. Average brightness 2002 is the average brightness of the virtual area of ​​the generated video data 301 calculated by the average brightness calculation unit 1702. It is preferable to render the average brightness 2001 and the average brightness 2002 using different modes (brightness, color, thickness, etc.). In the example shown in FIG. 20, the brightness is different. In the example shown in FIG. 20, the average brightness 2001 is darker than the average brightness 2002, so the setting of the imaging device 200 is originally than exposure This suggests that it is under.

[0056] Effects of this embodiment According to this embodiment, since the average brightness 2001 and the average brightness 2002 do not match, the user can intuitively know that there is an error in the settings of the image capture device 200 .

[0057] Fifth embodiment In the first embodiment, an example in which a first waveform monitor and a second waveform monitor are displayed is described. In the present embodiment, an example in which it is possible to switch between displaying the first waveform monitor and the second waveform monitor, or displaying only the first waveform monitor, depending on a setting operation by the user and the presence or absence of area information is described.

[0058] ●Signal processing device configuration The signal processing device 100 additionally has means for the user to perform setting operations, such as operation keys and an OSD menu. The signal processing device 100 has three display settings for the waveform monitor, "Standard", "Automatic", and "Extended", which the user can select.

[0059] ● Control flow of signal processing device FIG. 21 is a flowchart showing a control flow of the signal processing device 100, which is executed by the control unit 103 every time the receiving unit 101 acquires one frame of video data.

[0060] In S2101, if the waveform monitor display setting is "standard", the process transitions to S2104, and if not, to S2102. In S2102, if the waveform monitor display setting is "auto", the process transitions to S2103, and if not, to S2105. To summarize the operations of S2101 and S2102, if the waveform monitor display setting is "standard", the process transitions to S2104, if the waveform monitor display setting is "auto", the process transitions to S2103, and if the waveform monitor display setting is "extended", the process transitions to S2105. In S2103, if the detection unit 102 can extract area information from the video data, the process transitions to S2105, and if not, the process transitions to S2104. Note that the area information is superimposed on the video data as ANC data in SDI or InfoFrame in HDMI, and therefore cannot be extracted unless an external device superimposes the area information.

[0061] In S2104, the display unit 106 displays one waveform monitor of the entire region of the video data. At this time, the generation unit 107 generates the waveform monitor of the entire region of the video data acquired from the receiving unit 101 as a first waveform monitor. The control unit 103 synthesizes the first waveform monitor and the video data, and outputs the result to the display unit 106 as synthesized video data. The display unit 106 displays the synthesized video data including the first waveform monitor. FIG. 22 is a diagram showing an example of the display of synthesized video data when one waveform monitor of the entire region of the video data is displayed. The waveform monitor 2201 is the first waveform monitor generated by the generation unit 107, and is obtained by analyzing the entire region of the video data and visualizing the characteristics. The same reference numerals as in FIG. 5 are the same as those in FIG. 5. The waveform monitor 2201 is synthesized with the video data shown in FIG. 5 to produce the synthesized video data shown in FIG. 22.

[0062] In S2105, the display unit 106 displays a first waveform monitor corresponding to the virtual area of ​​the video data and a second waveform monitor corresponding to the real area. Since this is the same as the first embodiment, details will be omitted.

[0063] Variations In this embodiment, when area information can be extracted from the video data, a first waveform monitor corresponding to the virtual area of ​​the video data and a second waveform monitor corresponding to the real area are displayed. Also, an example has been described in which a waveform monitor of the entire area of ​​the video data is displayed when area information cannot be extracted from the video data. Here, when area information can be extracted from the video data, a waveform monitor of the entire area of ​​the video data may be further displayed. In this case, the display device further has one generating means for generating a waveform monitor.

[0064] Effects of this embodiment According to this embodiment, the second waveform monitor can be appropriately displayed in a scene where it is necessary, improving usability.

[0065] Sixth embodiment ●Signal processing device configuration Fig. 23 is a block diagram showing an example of the configuration of a signal processing device according to this embodiment. An image display device 2300 in Fig. 23 includes a receiving unit 2301, a receiving unit 2302, a green back synthesis unit 2309, a control unit 2303, a memory unit 2304, a synthesis unit 2305, a display unit 2306, a generating unit 2307, and a generating unit 2308.

[0066] The receiving unit 2301 acquires the first input moving image data, and outputs the first input moving image data to the green screen composition unit 2309. The first input moving image data is moving image captured of a performer performing in front of a green screen.

[0067] The receiving unit 2302 acquires second input moving image data and outputs the second input moving image data to the green screen composition unit 2309. The second input moving image data is a moving image shot separately from the first input moving image data. In this embodiment, the receiving units 2301 and 2302 acquire image data for each frame of a moving image from an external device. The receiving units 2301 and 2302 are input terminals that comply with standards such as SDI and HDMI. The external device is an imaging device, a playback device, or the like.

[0068] The green screen synthesis unit 2309 synthesizes the first input moving image data acquired from the receiving unit 2301 and the second input moving image data acquired from the receiving unit 2302, generates moving image data, and outputs the generated moving image data to the synthesis unit 2305, the generation unit 2307, and the generation unit 2308. Here, the moving image data is composed of pixel values ​​acquired from the second moving image data for pixels with a green color in the first moving image data, and pixel values ​​acquired from the first moving image data for other pixels. The green screen synthesis unit 2309 also generates region information, and outputs the region information to the control unit 2303, the generation unit 2307, and the generation unit 2308. Here, the region information is information in which pixels with a green color in the first input moving image data are assigned as a virtual region, and other pixels are assigned as a real region. The green screen synthesis unit 2309 can also adjust the image quality (brightness and color) of the first input moving image data and the second input moving image data individually.

[0069] The control unit 2303 controls the processing of each unit of the image display device 2300. The control unit 2303 is connected to each unit via a control bus, but the control bus is not shown in the figure because it is complicated. For example, the control unit 2303 is an arithmetic processing circuit that executes a program stored in the memory unit 2304 to control the processing of each block of the image display device 2300. In this embodiment, the control unit 2303 controls the composition unit 2305 based on area information output from the green back composition unit 239.

[0070] The memory unit 2304 stores programs, parameters, etc. The programs and parameters stored in the memory unit 2304 are read and written by each block of the image display device 2300.

[0071] The synthesis unit 2305 synthesizes the first waveform monitor output by the generation unit 2307 and the second waveform monitor output by the generation unit 2308 with the video data based on the control content output by the control unit 2303, to generate synthesized video data. Then, the synthesis unit 2305 outputs the synthesized video data to the display unit 2306.

[0072] The display unit 2306 displays on its display surface a moving image based on the composite moving image data output from the composition unit 2305. The display unit 2306 is, for example, a liquid crystal display unit having a liquid crystal panel and a backlight unit, or an organic EL display panel.

[0073] The generating unit 2307 acquires the video data output from the green screen synthesizing unit 2309, and also acquires the area information output from the green screen synthesizing unit 2309. Then, it generates a waveform monitor (first waveform monitor) related to pixels designated as a virtual area by the area information in the video data. More specifically, it extracts one frame's worth of data from the video data, and further extracts pixels designated as a virtual area from that data, and generates the results of analyzing them as the first waveform monitor.

[0074] The generating unit 2308 acquires the video data output from the green screen synthesizing unit 2309, and also acquires the area information output from the green screen synthesizing unit 2309. Then, it generates a waveform monitor (second waveform monitor) related to pixels designated as a real area by the area information in the video data. More specifically, it extracts one frame's worth of data from the video data, and further extracts pixels designated as a real area from that data, and generates the results of analyzing them as the second waveform monitor.

[0075] The operation of each unit has been described, and the procedure for displaying the first waveform monitor and the second waveform monitor on the display unit 2306 by integrating the operations will now be described. The receiving unit 2301 receives the first input moving image data, and the receiving unit 2302 receives the second input moving image data. The green screen synthesis unit 2309 synthesizes the first input moving image data and the second input moving image data to generate moving image data and area information. The data is then output to the control unit 2303, the generating unit 2307, and the generating unit 2308. The generating units 2307 and 2308 refer to the area information to generate the first waveform monitor and the second waveform monitor. The synthesizing unit 2305 synthesizes the first waveform monitor and the second waveform monitor into moving image data, and outputs the data to the display unit 2306 as synthesized moving image data. The display unit 2306 displays the synthesized moving image data including the first waveform monitor and the second waveform monitor.

[0076] Example of first and second input video data 24A is a diagram showing an example of the first input moving image data. A performer 2403 is positioned in front of a green screen 2401.

[0077] 24B is a diagram showing an example of the second input video data, which is a video of an outside scene 2411 and standing trees 2412.

[0078] ●An example of video data Fig. 25 is a diagram showing an example of moving image data. A performer 2403 is cut out from the first input moving image data shown in Fig. 24(A) and composited onto the second input moving image data shown in Fig. 24(B). Therefore, the moving image data is composed of the performer 2403, an outside scene 2411, and standing trees 2412.

[0079] An example of composite video data displayed on the display unit Fig. 26 is a diagram showing an example of the composite video data displayed by the display unit 2306. Waveform monitor 2601 is a first waveform monitor generated by the generation unit 2307, and visualizes the characteristics by analyzing the virtual area of ​​the video data shown in Fig. 25. Waveform monitor 2602 is a second waveform monitor generated by the generation unit 2308, and visualizes the characteristics by analyzing the real area of ​​the video data shown in Fig. 25. Items denoted by the same reference numerals as in Fig. 25 are the same as in Fig. 25. Waveform monitors 2601 and 2602 are composited with the video data shown in Fig. 25 to produce the composite video data shown in Fig. 26.

[0080] Effects of this embodiment According to this embodiment, when analyzing the virtual area and the real area separately, the user does not need to specify the area. By referring to the analysis results of the first waveform monitor and the second waveform monitor, the brightness distribution of the virtual area and the real area can be confirmed. In addition, in adjusting the image quality when compositing a green screen, it becomes easy to adjust the image quality difference between the first input video data and the second input video data (for example, to make the brightness of the first input video data and the second input video data approximately uniform).

[0081] Seventh embodiment The seventh embodiment of the present invention will be described below, but since this embodiment is a modified version of the first embodiment, the differences from the first embodiment will be mainly described, and the same contents as the first embodiment will be omitted as appropriate. In the first embodiment, a waveform monitor is synthesized with video data to display the synthesized video data, but in this embodiment, a frequency analysis graph is synthesized with video data to display the synthesized video data.

[0082] ●Analysis of image frequency characteristics In this embodiment, Using each pixel signal of the video data, the level of each frequency at a certain interval is obtained by using a fast Fourier transform (FFT) etc. By obtaining the number i, frequency freq(i) and level levelLA(i) in order from the lowest frequency, a frequency analysis graph according to the frequency characteristics of the image can be obtained.

[0083] ●Signal processing device configuration The generation unit 107 acquires the video data output from the receiving unit 101 and also acquires the region information output from the detection unit 102, and displays a frequency analysis graph for pixels in the video data that are designated as a virtual region by the region information.

[0084] The generation unit 108 acquires the video data output from the receiving unit 101 and also acquires the region information output from the detection unit 102, and displays a frequency analysis graph for pixels in the video data that are designated as real regions by the region information.

[0085] The receiving unit 101, the detecting unit 102, the control unit 103, the memory unit 104, the synthesizing unit 105, and the display unit 106 are the same as those in the first embodiment.

[0086] An example of composite video data displayed on the display unit 27 is a diagram showing an example of the composite video data displayed by the display unit 106. The frequency analysis graph 2701 is a frequency analysis graph generated by the generation unit 107, and is a graph in which the virtual area of ​​the shot video data is analyzed to visualize the spatial frequency characteristics. The frequency analysis graph 2702 is a frequency analysis graph generated by the generation unit 108, and is a graph in which the real area of ​​the shot video data is analyzed to visualize the spatial frequency characteristics. The area 2703 is the virtual area of ​​the shot video data, and the area 2704 is the real area of ​​the shot video data, but the specific designs are omitted.

[0087] The horizontal axis of frequency analysis graph 2701 and frequency analysis graph 2702 is spatial frequency, with the left being low frequency and the right being high frequency. The vertical axis is frequency, with the top being high frequency and the bottom being low frequency.

[0088] An example different from the example shown in Fig. 27 will be described. Fig. 28 is a diagram showing an example of composite moving image data displayed by display unit 106. A frequency analysis graph 2801 is obtained by analyzing a virtual area of ​​the shot moving image data and visualizing the features. A frequency analysis graph 2802 is obtained by analyzing a real area of ​​the shot moving image data and visualizing the features.

[0089] In the frequency analysis graph 2801, a graph is drawn using the first mode for the region where the spatial frequency is lower than the first threshold, and the second mode for the region where the spatial frequency is higher than the first threshold. In the example shown in FIG. 28, the second mode uses a lighter gray than the first mode. The same is true for the frequency analysis graph 2802. However, the threshold of the spatial frequency for switching between the first mode and the second mode is a second threshold that is lower in the frequency analysis graph 2801 corresponding to the virtual region than in the frequency analysis graph 2802 corresponding to the real region. For example, the spatial frequency for switching between the first mode and the second mode is 1 / 4 of the sampling frequency in the frequency analysis graph 2802 corresponding to the real region, and 1 / 16 of the sampling frequency in the frequency analysis graph 2801 corresponding to the virtual region, but is not limited thereto. The user of the signal processing device 100 may set each frequency via an operation unit (not shown).

[0090] In the frequency analysis graph 2801 corresponding to the virtual region, when there is a frequency drawn in the second manner, it indicates that there is a pixel focused in the virtual region. In the frequency analysis graph 2802 corresponding to the real region, when there is a frequency drawn in the second manner, it indicates that there is a pixel focused in the real region.

[0091] Effects of this embodiment According to this embodiment, when analyzing the virtual area and the real area separately, the user does not need to specify the area. By referring to the frequency analysis graph of the virtual area and the frequency analysis graph of the real area, it becomes easy to check the focus state (whether or not it is in focus) of the virtual area and the real area separately.

[0092] In addition, since the video data in the virtual area is generated by shooting the LED wall, moire caused by the pixels of the LED wall is a concern. By adjusting the focus while referring to the frequency analysis graph of the virtual area, moire caused by focusing on the LED wall against the photographer's intention can be reduced.

[0093] Furthermore, by drawing a frequency analysis graph of the virtual domain from a spatial frequency lower than the real domain in the second mode, the photographer can easily notice that he or she has focused on the LED wall, and moire caused by focusing on the LED wall against the photographer's intention can be further reduced.

[0094] Eighth embodiment The eighth embodiment of the present invention will be described below, but since this embodiment is a modification of the first embodiment, the differences from the first embodiment will be mainly described, and the same contents as the first embodiment will be omitted as appropriate. In this embodiment, when displaying the composite video data, a distance graph is also displayed in addition to the waveform monitor in the first embodiment.

[0095] Imaging device processing The imaging device 200 superimposes the distance from the imaging device 200 to the display device 300, which the user measures and sets in the imaging device 200, onto the captured video data 109. If the optical axis of the imaging device 200 and the display surface of the display device 300 are not perpendicular to each other, the user measures the shortest and longest values ​​of the distance from the imaging device 200 to the display device 300 and sets them as a distance range in the imaging device 200, which the imaging device 200 superimposes onto the captured video data 109.

[0096] The imaging device 200 measures the distance from the imaging device 200 to the performer 503 and superimposes it on the captured video data 109. The imaging device 200 superimposes, among the distances from the imaging device 200 to the object to be captured measured by the distance measuring unit 205, the distance corresponding to the area of ​​the captured video data 109 set by the user via an operation unit (not shown) as the distance from the imaging device 200 to the performer 503. If the distance of the area specified by the user varies depending on the pixels within the area, the shortest and longest values ​​of the distance are superimposed as the distance range. Note that the user sets the area in which the performer 503 is reflected.

[0097] The imaging device 200 superimposes the depth of field onto the captured video data 109. The depth of field is the distance range from the imaging device 200 to the object being photographed that can be considered to be in focus. The imaging device 200 calculates it from the focal length (angle of view) of the lens of the optical unit 201, the F-number, the sensor size of the imaging unit 202, and the like.

[0098] An example of composite video data displayed on the display unit Fig. 29 is a diagram showing an example of composite video data. In distance graph 2901, the horizontal axis indicates the distance at which image capture device 200 can set the focus (from the shortest shooting distance to infinity). Distance graph 2901 is an index showing the distance range from image capture device 200 to display device 300, the distance range from image capture device 200 to performer 503, and the depth of field centered on the distance at which image capture device 200 sets the focus. The same symbols as in Fig. 5 are used in Fig. 29 as well. In the captured video data shown in Fig. 5, The virtual region of the video data was analyzed and its features were visualized. Waveform Monitor 601, Analyzing the real area of ​​video data and visualizing its features The waveform monitor 602 and distance graph 2901 are combined to produce the combined video data shown in FIG.

[0099] Example of distance graph drawing 30 is a diagram showing details of the distance graph 2901. The virtual distance 3001 is the distance range from the imaging device 200 to the display device 300. The generation unit 107 acquires the distance range from the imaging device 200 to the display device 300 that is superimposed on the shot video data 109 by the imaging device 200, draws the virtual distance 3001, and outputs it to the synthesis unit 105. The real distance 3002 is the distance range from the imaging device 200 to the performer 503. The generation unit 108 acquires the distance range from the imaging device 200 to the performer 503 that is superimposed on the shot video data 109 by the imaging device 200, draws the real distance 3002, and outputs it to the synthesis unit 105. A depth of field drawing unit (not shown) of the signal processing device 100 acquires the depth of field superimposed on the captured video data 109 by the imaging device 200, draws the depth of field 3003, and outputs it to the synthesis unit 105. The synthesis unit 105 synthesizes the video data with the waveform monitor 601, waveform monitor 602, and distance graph 2901 (virtual distance 3001, real distance 3002, depth of field 3003), and outputs it to the display unit 106 as synthesized video data. The display unit 106 displays the synthesized video data.

[0100] The generation unit 107, the generation unit 108, and a depth of field generation unit (not shown) render the virtual distance 3001, the real distance 3002, and the depth of field 3003 in visually different modes. For example, the virtual distance 3001 is rendered in dark gray, the real distance 3002 in a medium-light gray, and the depth of field 3003 in dark gray. The generation unit 107 renders the virtual distance 3001 when the area information output by the detection unit 102 includes a virtual area. The generation unit 108 renders the real distance 3002 when the area information output by the detection unit 102 includes a real area.

[0101] An example different from the example shown in FIG. 30 will be described. FIG. 31 is a diagram showing details of the distance graph 2901. Of the depth of field 3103, the range where the distance overlaps with the virtual distance 3001 is drawn in the first mode (depth of field 3104). The range where the distance does not overlap with the virtual distance 3001 is drawn in the second mode (depth of field 3105). The first mode and the second mode only need to be visually different, and for example, the first mode may be drawn in red and the second mode in a medium-light gray. Items indicated with the same reference numerals as in FIG. 30 are the same as those in FIG. 30.

[0102] The virtual distance 3001 may be rendered in two manners. For example, the range of the virtual distance 3001 that overlaps with the depth of field 3003 is rendered in a first manner, and the range of the virtual distance 3001 that does not overlap with the depth of field 3003 is rendered in a second manner.

[0103] Effects of this embodiment According to this embodiment, when analyzing the virtual area and the real area separately, the user does not need to specify the area. By adjusting the focus while referring to the distance graph, it is possible to reduce the moire caused by focusing on the LED wall against the photographer's intention. Furthermore, by drawing the distance where the depth of field and the virtual distance overlap in different modes, it becomes easier for the photographer to notice that the LED wall has been focused on, and the moire caused by focusing on the LED wall against the photographer's intention can be further reduced.

[0104] The above embodiment includes the following configurations.

[0105] (Configuration 1) a first acquisition means for acquiring image data obtained by capturing an image of a subject and a background image displayed on a background display by an imaging device; an extraction means for extracting information on a background image region and a subject region from the image data; a first generating means for generating a first index indicating at least one of the characteristics of luminance, color, and spatial frequency of the background image area; a second generating means for generating a second index of the same characteristic as that of the first generating means among the characteristics of luminance, color, and spatial frequency of the subject region; A signal processing device comprising: a synthesis unit that synthesizes the first index, the second index, and the image data and outputs synthesized image data.

[0106] (Configuration 2) Further comprising a second acquisition means for acquiring a reference value according to a characteristic or setting of the background display, 2. The signal processing device according to configuration 1, wherein the synthesis means outputs the synthetic image data in which a reference value is plotted on the first index.

[0107] (Configuration 3) 3. The signal processing device according to claim 2, wherein the reference value is at least one of a luminance of 200 nit, a luminance of 203 nit, a maximum luminance of the background display, a luminance corresponding to a reflectance of 18%, and a luminance corresponding to a reflectance of 100%.

[0108] (Configuration 4) The signal processing device according to configuration 2, wherein the reference value is at least one of the chromaticity of an achromatic color of a background display, the chromaticity of a red color in the color gamut of the background display, the chromaticity of a green color in the color gamut of the background display, and the chromaticity of a blue color in the color gamut of the background display.

[0109] (Configuration 5) a third acquisition means for acquiring background video data; a third generating means for generating a third index having the same characteristics as those of the first generating means among luminance, color, and frequency characteristics of a region of the background image data corresponding to the background image region; 5. The signal processing device according to any one of configurations 1 to 4, wherein the synthesis means outputs the synthetic image data on which the third index is synthesized.

[0110] (Configuration 6) 6. The signal processing device according to any one of configurations 1 to 5, further comprising a conversion means for converting the background image data in accordance with an imaging range of the imaging device.

[0111] (Configuration 7) a first calculation means for calculating a first average value which is an average value of the same characteristics as those of the first generation means among the luminance, color, and frequency characteristics of the background image region from the image data acquired by the first acquisition means; a third acquisition means for acquiring background video data; a second calculation means for calculating a second average value which is an average value of the same characteristics as those of the first generation means among the luminance, color, and frequency characteristics of a region corresponding to a background image region in the background image data; 7. The signal processing device according to any one of configurations 1 to 6, wherein the synthesis means outputs the synthetic image data in which the first average value and the second average value are synthesized.

[0112] (Configuration 8) The method further includes a fourth generating means for generating a fourth index that indicates the same characteristics as those of the first generating means among the luminance, color, and frequency characteristics of the entire region of the image data acquired by the first acquiring means, 8. The signal processing device according to any one of configurations 1 to 7, wherein the synthesis means outputs the synthetic image data on which the fourth index has been synthesized.

[0113] (Configuration 9) When the extraction means has acquired information on the background image area and the subject area in the image data, the synthesis means synthesizes the first index, the second index, and the image data to output synthesized image data; The signal processing device according to configuration 8, wherein when the extraction fails to obtain information on the background image region and the subject region in the image data, the synthesis means synthesizes the fourth indicator and the image data to output synthetic image data.

[0114] (Configuration 10) When the extraction means has acquired information on the background image area and the subject area in the image data, the synthesis means synthesizes the first index, the second index, the fourth index, and the image data to output synthesized image data; The signal processing device according to configuration 8, wherein when the extraction means is unable to acquire information on the background image region and the subject region in the image data, the synthesis means synthesizes the fourth indicator and the image data to output synthetic image data.

[0115] (Configuration 11) a first acquisition means for acquiring image data obtained by capturing an image of a subject using an imaging device; a third acquisition means for acquiring background video data; a first synthesis means for synthesizing the image data and the background image data into one image data and extracting information on a background image region and a subject region in the image data; a first generating means for generating a first index indicating a characteristic of the image data of the background image region; A second generating means for generating a second index indicative of a characteristic of the image data of the subject region; A signal processing device comprising: a synthesis unit that synthesizes the first index, the second index, and the image data and outputs synthesized image data.

[0116] (Configuration 12) the first index and the second index are a graph visualizing a spatial frequency characteristic, The first generating means draws the graph in a manner different from each other on the low frequency side and the high frequency side of a first threshold value, the second generating means draws the graph in a manner different from each other on the low frequency side and the high frequency side of a second threshold value, 2. The signal processing device according to configuration 1, wherein the first threshold value is smaller than the second threshold value.

[0117] (Configuration 13) The signal processing device according to any one of configurations 1 to 12, wherein the synthesis means outputs synthetic image data obtained by synthesizing the image data with a fifth index indicating at least one of a distance range between the subject and the imaging device, a distance range between the background display and the imaging device, a distance at which the imaging device can set its focus, and a depth of field centered on the distance at which the imaging device is set to its focus.

[0118] (Configuration 14) 14. The signal processing device according to any one of configurations 1 to 13, wherein the first index and the second index are a waveform monitor, a vectorscope, or a chromaticity diagram.

[0119] (Configuration 15) 6. The signal processing device according to configuration 5, wherein the third indicator is a waveform monitor, a vectorscope, or a chromaticity diagram.

[0120] (Configuration 16) 9. The signal processing device according to configuration 8, wherein the fourth index is a waveform monitor, a vectorscope, or a chromaticity diagram.

[0121] (Configuration 17) 14. The signal processing device according to configuration 13, wherein the fifth index is a distance graph.

[0122] (Configuration 18) 18. The signal processing device according to any one of configurations 1 to 17, further comprising control means for controlling a display means to display the composite image data output by the synthesis means.

[0123] (Method 1) a first acquisition step of acquiring image data obtained by capturing an image of a subject and a background image displayed on a background display by an imaging device; an extraction step of extracting information on a subject area and a background image area in the image data; a first generation step of generating a first index indicative of at least one of luminance, color, and spatial frequency characteristics of the subject region; a second generation step of generating a second index having the same characteristics as those of the first generation step among the luminance, color, and spatial frequency characteristics of the background image region; A control method for a signal processing device, comprising: a synthesis step of synthesizing the first index, the second index, and the image data and outputting synthesized image data.

[0124] (Program 1) A program for causing a computer to execute each step of the control method described in Method 1.

[0125] (Other embodiments) Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0126] The present invention also includes cases where a software program that realizes the functions of the above-described embodiments is supplied to a system or device having a computer capable of executing the program directly from a recording medium or via wired / wireless communication, and the program is executed.

[0127] Therefore, the program code itself that is supplied to and installed in a computer to realize the functional processing of the present invention also realizes the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

[0128] In this case, so long as the program has the functions of a program, the form of the program is not important, such as object code, a program executed by an interpreter, or script data supplied to an OS.

[0129] The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory.

[0130] As a method for supplying the program, a method is also conceivable in which the computer program forming the present invention is stored in a server on a computer network, and a connected client computer downloads the computer program. [Explanation of symbols]

[0131] 101, 801, 1401, 2301, 2302 Receiver 102 Detection unit 103, 2303 Control section 104, 2304 Memory section 105, 2305 Synthesis section 106, 802, 2306 display section 107, 108, 702, 1403, 2307, 2308 generation section 201 Optics Department 202 Imaging unit 203 Region information overlay unit 204 Transmitter 205 Ranging section 701 Reference value acquisition unit 802 Reference value output section 1402 Conversion unit 1701, 1702 Average brightness calculation unit 2309 Green Screen Composite

Claims

1. a first acquisition means for acquiring image data obtained by capturing an image of a subject and a background image displayed on a background display by an imaging device; an extraction means for extracting information on a background image region and a subject region from the image data; a first generating means for generating a first index indicating at least one of the characteristics of luminance, color, and spatial frequency of the background image area; a second generating means for generating a second index of the same characteristic as that of the first generating means among the characteristics of luminance, color, and spatial frequency of the subject region; a synthesizing means for synthesizing said first index, said second index and said image data to output synthesized image data.

2. The method further includes a second acquisition means for acquiring a reference value according to a characteristic or setting of the background display, 2. The signal processing apparatus according to claim 1, wherein the synthesizing means outputs the synthetic image data in which a reference value is plotted on the first index.

3. 3. The signal processing device according to claim 2, wherein the reference value is at least one of a luminance of 200 nits, a luminance of 203 nits, a maximum luminance of the background display, a luminance corresponding to a reflectance of 18%, and a luminance corresponding to a reflectance of 100%.

4. 3. The signal processing device according to claim 2, wherein the reference value is at least one of the chromaticity of the achromatic color of the background display, the chromaticity of red in the color gamut of the background display, the chromaticity of green in the color gamut of the background display, and the chromaticity of blue in the color gamut of the background display.

5. a third acquisition means for acquiring background image data; a third generating means for generating a third index having the same characteristics as those of the first generating means among luminance, color, and frequency characteristics of a region of the background image data corresponding to the background image region; 2. The signal processing apparatus according to claim 1, wherein the synthesizing means outputs the synthesized image data in which the third index is synthesized.

6. 6. The signal processing device according to claim 5, further comprising a conversion means for converting the background image data in accordance with an imaging range of the imaging device.

7. a first calculation means for calculating a first average value which is an average value of the same characteristics as those of the first generation means among the luminance, color, and frequency characteristics of a background image region in the image data acquired by the first acquisition means; a third acquisition means for acquiring background image data; a second calculation means for calculating a second average value which is an average value of the same characteristics as those of the first generation means among the luminance, color, and frequency characteristics of a region corresponding to a background image region in the background image data; 2. The signal processing apparatus according to claim 1, wherein said synthesis means outputs the synthetic image data in which said first average value and said second average value are synthesized.

8. The method further includes a fourth generating means for generating a fourth index that indicates the same characteristics as those of the first generating means among the luminance, color, and frequency characteristics of the entire region of the image data acquired by the first acquiring means, 2. The signal processing apparatus according to claim 1, wherein the synthesizing means outputs the synthesized image data in which the fourth index is synthesized.

9. When the extraction means has acquired information on the background image area and the subject area in the image data, the synthesis means synthesizes the first index, the second index, and the image data to output synthesized image data; The signal processing device according to claim 8, characterized in that, when the extraction fails to obtain information on the background image area and the subject area in the image data, the synthesis means synthesizes the fourth indicator and the image data to output synthetic image data.

10. When the extraction means has acquired information on the background image area and the subject area in the image data, the synthesis means synthesizes the first index, the second index, the fourth index, and the image data to output synthesized image data; 9. The signal processing device according to claim 8, characterized in that, when the extraction means is unable to obtain information on the background image area and the subject area in the image data, the synthesis means synthesizes the fourth indicator and the image data to output synthetic image data.

11. a first acquisition means for acquiring image data obtained by capturing an image of a subject using an imaging device; a third acquisition means for acquiring background image data; a first synthesis means for synthesizing the image data and the background image data into one image data and extracting information on a background image region and a subject region from the image data; a first generating means for generating a first index indicating a characteristic of image data of the background image region; a second generating means for generating a second index indicative of a characteristic of the image data of the subject region; a synthesizing means for synthesizing said first index, said second index and said image data to output synthesized image data.

12. the first index and the second index are graphs that visualize spatial frequency characteristics, The first generating means draws the graph in a manner different from each other on the low frequency side and the high frequency side of a first threshold value, the second generating means draws the graph in a manner different from each other on the low frequency side and the high frequency side of a second threshold value, The signal processing device according to claim 1 , wherein the first threshold value is smaller than the second threshold value.

13. The signal processing device according to claim 1, characterized in that the synthesis means outputs synthetic image data obtained by synthesizing the image data with a fifth index indicating at least one of the distance range between the subject and the imaging device, the distance range between the background display and the imaging device, the distance at which the imaging device can set its focus, and the depth of field centered on the distance at which the imaging device is set to its focus.

14. 14. The signal processing device according to claim 1, wherein the first index and the second index are a waveform monitor, a vectorscope, or a chromaticity diagram.

15. 6. The signal processing device according to claim 5, wherein the third indicator is a waveform monitor, a vectorscope, or a chromaticity diagram.

16. 9. The signal processing device according to claim 8, wherein the fourth indicator is a waveform monitor, a vectorscope, or a chromaticity diagram.

17. The signal processing device according to claim 13, wherein the fifth index is a distance graph.

18. 14. The signal processing apparatus according to claim 1, further comprising a control unit that controls a display unit to display the composite image data output by the combining unit.

19. a first acquisition step of acquiring image data obtained by capturing an image of a subject and a background image displayed on a background display by an imaging device; an extraction step of extracting information on a subject area and a background image area in the image data; a first generation step of generating a first index indicative of at least one of luminance, color, and spatial frequency characteristics of the subject region; a second generation step of generating a second index having the same characteristics as those of the first generation step among the characteristics of luminance, color, and spatial frequency of the background image region; A method for controlling a signal processing device, comprising: a synthesis step of synthesizing the first index, the second index and the image data to output synthetic image data.

20. A program for causing a computer to execute each step of the control method according to claim 19.

Images