Information processing methods, information processing systems, and programs

Abstract

[Problem] To suppress the appearance of light emanating from the outline of an object being photographed. [Solution] One aspect of this technology involves an information processing method that displays a screen showing an image of the object as its background, indicating that the amount of light in the area overlapping the object on the camera's image covering at least a portion of the screen's display surface has been reduced. This technology is applicable to virtual filming using large LED displays.

Description

Technical Field

[0001] This technology relates to information processing methods, information processing systems, and programs, and in particular to information processing methods, information processing systems, and programs that can suppress the apparent glowing of the outline of an object as a subject. Prior Technology

[0002] Recently, the use of large LED (Light Emitting Diode) displays in film and television production, also known as virtual production, has become widely adopted. Virtual production involves filming actors performing in front of an LED display showing an image as a background.

[0003] This allows for filming even when photographers or actors cannot be present on set. [Previous Technical Documents] [Patent Literature]

[0004] [Patent Document 1] International Publication No. 2018 / 164105 Summary of the Invention

[0005] [The problem that the invention aims to solve]

[0006] Since LED displays are self-emissive devices, during virtual filming, the light emitted by the LED display will shine from behind the subject, i.e., the actor.

[0007] An actor being lit from behind, when viewed from the front, may sometimes appear to have a glowing outline. An image captured under these conditions will be an image not intended by the photographer.

[0008] This technology was developed in view of this situation, with the aim of suppressing the apparent glow of the outline of the object being photographed. [Methods used to solve problems]

[0009] One aspect of this technology is an information processing method that, for a display showing an image of an object as a background, displays a photographic image of a camera covering at least a portion of the display surface of the prior object, in which the light intensity of the area overlaid by the prior object has been reduced.

[0010] In one aspect of this technology, for a display that shows an image as the background of an object, a display image in which the light intensity of the area overlaid by the object has been reduced on the photographic image of a camera covering at least a portion of the display surface of the prior display. Simple Explanation of the Diagram

[0011] [Figure 1] An illustration of the photographic appearance of the photographic system described in one embodiment of this technology. [Figure 2] An illustration of an example of a light source used in virtual filmmaking photography. [Figure 3] An illustration of an example of a light source in typical photography. [Figure 4] View from above showing the illumination caused by the LED display. [Figure 5] View of the illumination caused by the LED display from the front. [Figure 6] An illustration of an example of an LED display image. [Figure 7] shows an example of image generation. [Figure 8] A diagram illustrating the illumination pattern of a situation where the light intensity of an image in a reduced light intensity area is used as a display image. [Figure 9] Block diagram of the hardware configuration example of the PC in Figure 1. [Figure 10] Block diagram of a PC's functional configuration example. [Figure 11] A diagram illustrating the composition of the display surface of an LED display. [Figure 12] An illustration of an example of object identification. [Figure 13] Illustration of another example of object identification. [Figure 14] An illustration of an example of a determination of the area of ​​reduced light intensity. [Figure 15] Illustration of another example of the determination of the field of light reduction. [Figure 16] Flowchart illustrating PC processing. [Figure 17] An illustration of an example of adjusting the light intensity in a region of reduced light intensity. [Figure 18] Illustration of an example of the structure of an LED display. [Figure 19] shows a block diagram of an example of the structure of a unit. Implementation

[0012] The following describes the form in which this technology is implemented. The description is presented in the following order. 1. About Virtual Production and Cinematography 2. Regarding the images used in LED displays. 3. Composition of PC 4. PC Actions 5. Adjustment of light intensity in areas of reduced light intensity 6. Regarding LED displays 7. Other

[0013] <1. About Virtual Production and Cinematography> Example of a photographic system Figure 1 is an illustration of the photographic appearance of the photographic system 1 described in one embodiment of the present technology.

[0014] The photography system 1 in Figure 1 mainly consists of a PC 11, an LED display 12, and a camera 13. The photography system 1 also includes other equipment such as a monitor for image inspection and cameras other than the camera 13.

[0015] Camera System 1 is set up in a spacious space such as a film studio. With Camera System 1, filming of movies or TV series can be carried out through so-called virtual production photography.

[0016] In the photography studio, a large, wall-shaped LED display 12 is installed. As described later, the LED display 12 is constructed by combining a plurality of display units into a tile-like shape. Through the images displayed by the plurality of display units, a huge image is displayed on the entire display surface of the LED display 12. In the example of Figure 1, an image of a landscape with buildings along a road is displayed.

[0017] As shown in Figure 1, actor H stands in front of LED display 12, performing with the image displayed on LED display 12 as the background. The image displayed on LED display 12 is the background image (background image) for actor H.

[0018] Virtual filming is performed by photographing actor H and a background image using a camera 13. In the example of Figure 1, the camera 13, which is positioned facing actor H, is mounted at the tip of a cantilever.

[0019] The image captured by the camera 13 includes at least a portion of both the actor H and the background image, depending on the camera's field of view (angle). By changing the background image as the actor H moves or the camera 13 moves, an image that appears to exist in the space projected by the LED display 12 can be captured.

[0020] PC 11 is a computer used for the display of LED display 12 or for the control of photography caused by camera 13. An information processing system is constituted by PC 11. The functions of PC 11 can be implemented by one computer or by multiple computers. Furthermore, at least some of the functions of PC 11 can also be implemented by a server on the Internet.

[0021] • Regarding the lighting environment during photography Figure 2 is an illustration of an example of a light source used in virtual filmmaking photography.

[0022] The camera 13 is positioned facing the actor H and the display surface of the LED display 12. The hollow arrow in Figure 2 indicates the direction of the camera's field of view.

[0023] As shown by arrows #1 and #3 in Figure 2, the light illuminating actor H during virtual filming consists of three types: light emitted from the back side by LED display 12, light emitted from above by lighting machine 14, and light emitted from the front side by lighting machine 15. Lighting machine 14 is, for example, a lighting machine installed on the ceiling of a film studio. Lighting machine 15 is, for example, a vertical lighting machine prepared near the filming camera 13. Alternatively, an LED display, different from LED display 12, can be used as a lighting machine and installed on the ceiling or near the filming camera 13, illuminating actor H from above or the front side using the light emitted from the LED display for lighting purposes.

[0024] On the other hand, during normal outdoor shooting in non-virtual production, the light illuminating actor H, as shown in Figure 3, consists of two types of light: natural light such as sunlight and light emitted by lighting equipment prepared near the camera. If we assume that the light from lighting equipment 14, such as the ceiling, is equivalent to natural light during normal shooting, then during virtual production, in addition to the light during normal shooting, the light emitted by LED display 12 will also illuminate actor H.

[0025] Figure 4 is a top view of the illumination caused by the LED display 12.

[0026] Since the LED display 12 is a self-emissive device, as shown in the schematic diagram in Figure 4, the light emitted by the LED display 12 shines on the actor H from the rear side. As shown in Figure 5A, a portion of the light from the LED display 12 that shines on the actor H will travel along the actor H's body towards the front side (the side of the camera 13).

[0027] Viewed from the front side, as shown in Figure 5B, the outline of actor H appears to glow due to light from the rear side. In Figure 5B, thin dots are depicted around the outline of actor H to indicate that the outline is glowing.

[0028] Aside from situations where the light source is intentionally placed on the side of the LED display, such as backlit scenes, images taken in this condition will appear to glow due to the actors' outlines, resulting in images that are not intended by the photographer.

[0029] In one embodiment of the photographic system 1, the image required to suppress the appearance of light emanating from the outline of actor H is generated in PC 11 and used for display on LED display 12.

[0030] <2. Regarding the images used in LED displays> Figure 6 is an illustration of an example of the display image of the LED display 12.

[0031] The box F in the upper part of Figure 6 represents the shooting range of the camera 13. In the image captured by the camera 13, the actor H's approximate full body is shown at a position slightly to the right of the center.

[0032] In this case, as shown in the lower part of Figure 6, the area overlapping the actor H in the photographic image within the entire background image is designated as the light reduction area A1. The image in which the light reduction area A1 has reduced light is displayed as the display image on the LED display 12. In the lower part of Figure 6, the light reduction area A1 located on the right side of the photographic range is illustrated in black, indicating that this is an area with less light than the surrounding area.

[0033] Thus, the light reduction area A1 is determined based on the positional relationship between the camera 13 and the actor H. The image in which the light reduction area A1, within the entire background image, has had its light reduced, is displayed as a display image on the LED display 12. In the example of Figure 6, although the shape of the light reduction area A1 is set as a long rectangular area, it can also be set as other shapes. For example, an area with the same shape as the outline of actor H, determined along the outline of actor H, can also be determined as the light reduction area A1.

[0034] Figure 7 is an illustration of an example of image generation.

[0035] If photography is performed as described with reference to FIG6, the image shown in the upper part of FIG7 will be captured. Based on the image captured by the camera 13, the actor H, who is the subject, will be identified. As shown in the middle part of FIG7, the position of actor H in the image is identified. In the middle part of FIG7, the area indicated by the diagonal lines represents the area reflected by actor H.

[0036] Once the area reflected by actor H has been identified, the corresponding region on the background image is determined as a light reduction region A1, as shown in the lower part of Figure 7. In PC11, the image with reduced light intensity in the light reduction region A1 within the entire background image is generated as a display image and used for display on the LED display 12. For example, by changing the color of the light reduction region A1 to black, the light intensity of the light reduction region A1 will be suppressed.

[0037] Figure 8 is a diagram illustrating the illumination pattern of a display image in a region A1 where the light intensity has been reduced and the image is being used as the display image.

[0038] When an image showing reduced light intensity in the light reduction area A1 is displayed on the LED display 12, as shown in Figure 8A, the light directly hitting the actor H from the rear side is suppressed. This prevents the light emitted by the LED display 12 from traveling around the actor H's body to the front side.

[0039] When viewed from the front side, the light from the rear side is suppressed, thus preventing the outline of actor H from appearing to glow, as shown in Figure 8B. In Figure 8B, the lack of color around the outline of actor H indicates that the outline is not glowing. The image captured in this state is the image of actor H with the outline not glowing.

[0040] Thus, by using an image of the area corresponding to actor H, where the light intensity has been reduced, on the LED display 12, the apparent glow of actor H's outline can be suppressed. The photographer can then capture the desired image of actor H's outline without any glow.

[0041] The operation of PC11, which controls the display of LED display 12 as described above, will be described later with reference to the flowchart.

[0042] <3. PC Composition> Figure 9 is a block diagram of an example of the hardware configuration of PC11 in Figure 1.

[0043] In PC11 of Figure 9, CPU (Central Processing Unit) 31, ROM (Read Only Memory) 32, and RAM (Random Access Memory) 33 are interconnected via bus 34.

[0044] An input / output interface 35 is connected to the bus 34. An input section 36 consisting of a keyboard, mouse, etc., and an output section 37 consisting of a display, speaker, etc. are connected to the input / output interface 35.

[0045] Furthermore, the input / output interface 35 is connected to a memory unit 38, which is made of hard disk or non-volatile memory, a communication unit 39, which is made of network interface, and a drive 40 for driving removable media 41.

[0046] The communication unit 39 communicates with the controller that controls the display of the LED display 12 or the camera 13. For example, the communication unit 39 receives photographic images captured by the camera 13, or camera information containing photographic parameters such as the position and shooting range of the camera 13. This information is transmitted, for example, from the camera 13 or a device that controls the operation of the camera 13.

[0047] Figure 10 is a block diagram of a functional configuration example of PC11.

[0048] As shown in Figure 10, an information processing unit 51 is implemented in the PC 11. The information processing unit 51 is composed of an image acquisition unit 61, a recognition unit 62, a light intensity reduction area determination unit 63, a display image generation unit 64, and a display control unit 65. At least a portion of the functional units shown in Figure 10 are implemented by executing a predetermined program by the CPU 31 of Figure 9, which constitutes the PC 11.

[0049] Image acquisition unit 61 acquires the background image displayed on LED display 12. For example, the background image is acquired by reading image data from memory unit 38 of PC 11. Image acquisition unit 61 also controls communication unit 39 to acquire photographic images transmitted from camera 13 or the like. The background image and photographic image acquired by image acquisition unit 61 are output to recognition unit 62. The background image is also output to display image generation unit 64.

[0050] The recognition unit 62 identifies objects such as actor H located in front of the LED display 12, or objects placed in front of the LED display 12, based on the images supplied from the image acquisition unit 61. During filming, objects such as film sets are sometimes placed near actor H.

[0051] • Object identification method 1 The identification unit 62 analyzes the photographic image and performs person or object recognition to identify objects. It can also identify objects by calculating the difference between the photographic image and the background image corresponding to the photographic area.

[0052] • Object identification method 2 Figure 11 is a diagram illustrating the configuration of the display surface of the LED display 12.

[0053] As shown in Figure 11, a plurality of sensors, such as infrared sensors, can be arranged on approximately the entire display surface of the LED display 12. The small black dots in Figure 11 represent the sensors.

[0054] The identification unit 62 acquires the measurement results from each sensor and identifies objects located in front of the LED display 12. For example, in the case where actor H and object O are located in front of the LED display 12 as objects as shown in FIG12, each object is identified based on the measurement results from the sensors.

[0055] ·Object identification method 3 Figure 13 is an illustration of another example of object identification.

[0056] As shown in Figure 13A, the cameras that are different from the photography camera 13, namely the identification cameras 81-1 and 81-2, can also be set so that their shooting range faces in front of the LED display 12. In the example of Figure 13A, the identification cameras 81-1 and 81-2 are set above the LED display 12.

[0057] The recognition unit 62 identifies objects such as actor H or object O located in front of the LED display 12 by analyzing the images captured by the recognition cameras 81-1 and 81-2, as shown in Figure 13B.

[0058] The information of the object thus identified is output to the light reduction area determination unit 63 in Figure 10. Information such as the position and size of the object is provided to the light reduction area determination unit 63.

[0059] The light reduction area determination unit 63 determines the light reduction area based on information about the object supplied from the identification unit 62. When determining the light reduction area, the position or shooting range of the camera 13, specified by camera information, is appropriately used. The light reduction area determination unit 63 is supplied with camera information received in the communication unit 39.

[0060] Figure 14 is an illustration of an example of the determination of the light intensity reduction area caused by the light intensity reduction area determination unit 63.

[0061] The light reduction area determination unit 63, for example, performs ray tracking based on the positional relationship between the camera 13 and the actor H, and determines the light reduction area A1.

[0062] As shown in Figure 14, the ray tracing system establishes a straight line L connecting the position P1 of the camera 13 and the position P2 on the outline of the actor H, and detects the position P3 on the display surface of the LED display 12 that intersects the straight line L. By performing the same processing on each position on the outline of the actor H, the area on the display surface of the LED display 12 corresponding to the area reflected by the actor H in the photographic image is identified. The light reduction area A1 is determined in such a way that it includes at least a portion of the area on the display surface of the LED display 12 that has been identified in this way.

[0063] Alternatively, a region with the same shape as the region reflected by actor H in the photographic image can be designated as a region of reduced light intensity, or a region with a predetermined shape such as a rectangle can be designated as a region of reduced light intensity.

[0064] Furthermore, the light reduction area determination unit 63 can also determine the light reduction area based on information specified by the photographer.

[0065] Figure 15 is an illustration of another example of the determination of the light intensity reduction area caused by the light intensity reduction area determination unit 63.

[0066] The display screen S shown in FIG15 is displayed on the display unit of the tablet terminal (not shown) or PC11 installed in the camera system 1. The display screen S is the screen used when specifying the area of ​​reduced light intensity. As shown in FIG15, the display screen S includes the following as constituent elements of the screen: area specification unit E1, brightness specification unit E2, and detection object display unit E3.

[0067] The photographer can instruct various elements of the displayed screen S by operating a touch panel, mouse, keyboard, etc. The tablet terminal or PC 11 sends the specified information generated based on the photographer's operation of the various elements of the displayed screen S to the light reduction area determination unit 63. The specified information sent to the light reduction area determination unit 63 is information indicating the content of the photographer's operation. The light reduction area determination unit 63 determines the position and range of the light reduction area or the brightness of the light reduction area based on the specified information sent from the tablet terminal or PC 11.

[0068] The area designation unit E1 on the left side of the display screen S is an element of the display screen S set up to designate the position and range of the light reduction area A1. Based on the position and range of the light reduction area A1 designated by the photographer using the area designation unit E1, the designation information required to designate the position and range of the light reduction area A1 is generated. The generated designation information is then sent to the light reduction area determination unit 63. Alternatively, the designation may not involve both the position and range of the light reduction area, but only either the position or the range, and this designation may be performed using the display of the area designation unit E1.

[0069] As shown in Figure 15, grid lines can also be displayed in the designated area E1. The designated area E1 can also be designated based on the grid lines.

[0070] Furthermore, the position and range of the light reduction area A1 determined by the light reduction area determination unit 63 based on the recognition result of the recognition unit 62 can also be displayed as an initial value in the area designation unit E1. In this case, the photographer changes the position and range of the light reduction area A1 determined by the light reduction area determination unit 63 as needed.

[0071] The brightness specification unit E2 above the display screen S is an element of the display screen S set up to specify the brightness of the light reduction area A1. When the photographer uses the brightness specification unit E2 to specify the brightness of the light reduction area A1, the specification information required to specify the brightness of the light reduction area A1 is generated. The generated specification information is then sent to the light reduction area determination unit 63.

[0072] As shown in Figure 15, the brightness specification section E2 can also be constructed by sliding the slider as needed. The brightness of the light reduction area A1 can also be specified by operating the slider.

[0073] Furthermore, the brightness of the light reduction zone A1 set by the light reduction zone determination unit 63 based on the recognition result of the recognition unit 62 can also be displayed as an initial value in the brightness specification unit E2. In this case, the photographer changes the brightness of the light reduction zone A1 determined by the light reduction zone determination unit 63 as needed.

[0074] The object detection display unit E3 on the right side of the display screen S is an element of the display screen S set up to indicate objects contained in the light reduction area A1. In the example of Figure 15, actor H is indicated as an object.

[0075] Based on the recognition results of the recognition unit 62, the object will be displayed on the object detection display unit E3. By operating the object detection display unit E3, the photographer can select which object to specify the light reduction area A1 from among the multiple objects identified by the recognition unit 62.

[0076] Furthermore, the scope (outline) of an object can be corrected based on the recognition results of the recognition unit 62 by operating the object detection display unit E3.

[0077] The object recognition by the recognition unit 62 and the determination of the light reduction area by the light reduction area determination unit 63 are performed, for example, on each frame of the photographic image and the background image. At least one of the position and range of the light reduction area changes with the movement of actor H or the shooting range of the camera 13. The information of the light reduction area determined by the light reduction area determination unit 63 is output to the display image generation unit 64.

[0078] The display image generation unit 64 generates a display image by reducing the light intensity of the light intensity reduction area determined by the light intensity reduction area determination unit 63 within the entire background image supplied from the image acquisition unit 61.

[0079] For example, the display image generation unit 64 generates a background image with the color set to black in the light reduction area as a display image. A background image in the light reduction area where the brightness has been reduced can also be generated as a display image, or brightness information can be removed from the pixel information of each pixel constituting the light reduction area, leaving only color information, and then generated as a display image. The display image generated by the display image generation unit 64 is output to the display control unit 65.

[0080] The display control unit 65 displays a display image in which the light intensity of the area where the object overlaps (i.e., the area of ​​reduced light intensity) has been reduced. The display control unit 65 controls the communication unit 39 to send the display image supplied from the display image generation unit 64 to a controller (not shown) of the LED display 12, thereby causing the LED display 12 to display.

[0081] <4. PC Actions> Here, the operation of PC11 is explained with reference to the flowchart in Figure 16.

[0082] In step S1, the image acquisition unit 61 acquires the background image currently being displayed on the LED display 12 and the photographic image captured by the camera 13.

[0083] In step S2, the identification unit 62 identifies the object located in front of the LED display 12.

[0084] In step S3, the identification unit 62 determines whether an object located in front of the LED display 12 is reflected in the photographic image of the camera 13, and waits until it is determined that the object is reflected in the photographic image of the camera 13. If the object located in front of the LED display 12 is not reflected in the photographic image of the camera 13, the background image that has not been set to a light reduction area is directly used as the display image of the LED display 12.

[0085] If it is determined in step S3 that an object located in front of the LED display 12 is reflected in the photographic image of the photographic camera 13, then in step S4, the light reduction area determination unit 63 determines the light reduction area as described above.

[0086] In step S5, the display image generation unit 64 generates an image in which the light intensity of the light intensity reduction area in the entire background image has been reduced, as a display image.

[0087] In step S6, the display control unit 65 causes the display image generated by the display image generation unit 64 to be displayed on the LED display 12. The display image in which the light intensity of the light reduction area has been reduced will continue, for example, during the period when photography is being performed by the camera 13.

[0088] Through the above processing, the image with reduced light intensity in the light reduction area is used for display on LED display 12. PC11 is used to photograph the image that suppresses the outline of actor H from appearing to glow.

[0089] <5. Adjustment of light intensity in areas of reduced light intensity> Alternatively, the area with reduced light intensity may not be set to a completely dark area (a black area), but rather the light intensity may be adjusted according to various conditions. The adjustment of light intensity is performed by the display image generation unit 64 during the generation of the display image.

[0090] • Adjustment Example 1 Figure 17 is an illustration of an example of adjusting the light intensity in a region of reduced light intensity.

[0091] The amount of light can also be adjusted to reduce the amount of light in the area according to the positional relationship between the object and the LED display 12.

[0092] For example, if the distance between actor H and LED display 12 is relatively close, as shown in Figure 17A, then compared to a more distant distance, as shown in Figure 17B, the light intensity in the light reduction area can be adjusted by increasing the reduction amount. The fact that the light reduction area A1-1 in Figure 17A is shown in black, and the light reduction area A1-2 in Figure 17B is shown in a lighter color, indicates that the light reduction area A1-1 is darker (the reduction amount is greater) than the light reduction area A1-2.

[0093] Generally, the closer an object is to the LED display 12, the more light from the back side will illuminate the object, making the outline of the object more noticeable. By adjusting the amount of light in the light reduction area so that the closer the object is to the LED display 12, the more light will be reduced, thus displaying a background image with less anomaly while suppressing the amount of light emitted from the outline.

[0094] • Adjustment Example 2 The amount of light can also be adjusted to reduce the amount of light in the area, depending on the photographic scene. In this case, information about the photographic scene is supplied to the display image generation unit 64.

[0095] For example, in scenes where the actor's silhouette appears to glow naturally, such as backlit scenes, the amount of light reduction in the light reduction area is set to a smaller amount. In this way, the appearance of the actor's silhouette appearing to glow can be reproduced.

[0096] • Adjustment Example 3 Alternatively, the light intensity can be adjusted to reduce the light intensity in the area according to the intensity of the light illuminating the object. In this case, information about the light intensity is supplied to the display image generation unit 64.

[0097] For example, when the lighting on the actor is strong, the amount of light reduction in the area where the light is reduced should be set to a smaller amount compared to when the lighting on the actor is weak.

[0098] Generally, when the lighting on an object is stronger, the glow in the outline becomes less noticeable compared to when it is weaker. By adjusting the amount of light reduction in a stronger lighting environment, the amount of light in the outline can be reduced, which can both display a less unusual background image and suppress the glow in the outline.

[0099] • Adjustment Example 4 The amount of light in the reduced area can also be adjusted according to the focus position of the camera 13. In this case, the camera information supplied to the display image generation unit 64 will include information about the focus position.

[0100] For example, the amount of light in the reduced area is adjusted so that the more the camera 13 focuses on the actor, the more the light is reduced.

[0101] Generally, the more accurately the focus is aligned, the more prominent the glow of the outline becomes. By adjusting the amount of light reduction in the area where the focus is aligned more accurately, it is possible to display a background image with less anomaly while suppressing the amount of glow in the outline.

[0102] <6. About LED Displays> Figure 18 is a diagram illustrating an example of the configuration of the LED display 12.

[0103] The LED display 12 is constructed by arranging a plurality of display units, namely display units (racks) 91-1 to 91-n, in a tile-like shape. Hereinafter, unless it is necessary to distinguish each display unit 91-1 to 91-n individually, they will be collectively referred to as display unit 91.

[0104] Each display unit 91, as shown in the enlarged frame, is constructed by arranging LED arrays 101, which are modules in a matrix (2D array) configuration corresponding to each pixel, into a tile-like shape. In the example of Figure 18, one display unit 91 is constructed by 4×3 LED arrays 101. Hereinafter, unless it is necessary to distinguish between each LED array 101-1 and 101-m, they will be collectively referred to as LED array 101.

[0105] For example, the object detection sensors described with reference to FIG11 are respectively disposed on the LED array 101.

[0106] As shown in Figure 18, each display unit 91 of the LED display 12 is connected to an external device, namely the LED display controller 111. The functions of the LED display controller 111 can also be integrated into the aforementioned PC 11, thus enabling the LED display controller 111 to be configured as an external device of the PC 11. The generation of the aforementioned display image can also be performed in a separate device from the PC 11, namely the LED display controller 111.

[0107] The LED display 12 displays an image corresponding to a frame unit based on an image signal supplied from the LED display controller 111. Specifically, the image signal supplied to each display unit 91 is an image signal corresponding to the position of that display unit 91 on the LED display 12. Each display unit 91 displays an image based on the image signal, thereby the entire LED display 12 displays an image of one frame.

[0108] The LED display controller 111 performs predetermined signal processing on the image signal of the frame unit supplied from the external machine. The LED display controller 111 divides the resulting image signal of the frame unit into n units according to the position of each display unit 91 and sends them to each display unit 91.

[0109] Figure 19 is a block diagram of a configuration example of display unit 91.

[0110] As shown in Figure 19, the display unit 91 is composed of a driver control unit 121 and an LED block 122.

[0111] The driver control unit 121 consists of: signal input IF131, signal processing unit 132, and signal output IF133-1 and IF133-m.

[0112] The signal input IF131 receives the image signal of the frame unit sent from the LED display controller 111 and supplies it to the signal processing unit 132.

[0113] The signal processing unit 132 performs signal processing necessary for its own display unit 91, such as color correction or brightness correction, on the image signal of the frame unit supplied from the signal input IF 131. The signal processing unit 132 divides the image signal of the frame unit after signal processing into m segments according to the positions of each LED array 101 on its own display unit 91. The signal processing unit 132 generates the display signal required to set the luminous intensity of each LED constituting each LED array 101 based on the image signal of the frame unit after signal processing.

[0114] The signal processing unit 132 is connected to signal outputs IF133-1 to IF133-m corresponding to each of the LED arrays 101-1 to 101-m. The signal processing unit 132 supplies the display signals of each LED array 101 to the signal outputs IF133-1 to IF133-m corresponding to that LED array 101.

[0115] The signal outputs IF133-1 and IF133-m respectively send the display signals supplied from the signal processing unit 132 to the LED block 122.

[0116] LED block 122 is composed of LED drivers 141-1 and 141-m and LED arrays 101-1 and 101-m. LED drivers 141-1 and 141-m are respectively connected to LED arrays 101-1 and 101-m.

[0117] LED drivers 141-1 and 141-m drive and control the light emission of each LED constituting LED array 101-1 and 101-m based on the display signals sent from signal outputs IF133-1 and 133-m.

[0118] <7. Other> Although the LED display 12 with the above configuration is installed in a ground-standing state (with the display surface set to a vertical plane), the LED display 12 can also be installed on the ground. In this case, the LED display 12 is installed with the display surface facing upwards.

[0119] Similar to the case where the LED display 12 is set up on the ground, even when set up on the ground, a display image in which the amount of light in the area where the object overlaps in the photographic image has been reduced will be appropriately generated and used for display on the LED display 12 set up on the ground.

[0120] The LED display 12 can also be installed on the ceiling surface or on multiple surfaces surrounding actor H.

[0121] About the program The aforementioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program that constitutes the software is installed on a computer with embedded dedicated hardware or a general-purpose personal computer, etc.

[0122] The programs executed by the computer can be programs that process data in a time sequence according to the order described in this manual, or programs that process data in a parallel or concurrent manner as necessary.

[0123] The effects described in this instruction manual are for illustrative purposes only and are not limited; other effects may also be possible.

[0124] The implementation of this technology is not limited to the above-mentioned implementation forms, and various changes can be made within the scope of the main idea of ​​this technology.

[0125] For example, this technology can also distribute one function across multiple devices via a network, using cloud computing to perform common processing.

[0126] Furthermore, the steps described in the above flowchart can be performed by one device or by multiple devices.

[0127] Furthermore, if a step contains multiple processes, those processes can be performed by one device or by multiple devices.

[0128] In this specification, the term "system" means a collection of multiple constituent elements (devices, modules (parts), etc.), regardless of whether all constituent elements are located in the same enclosure. Therefore, multiple devices housed in individual enclosures and connected via a network, and a single device containing multiple modules in a single enclosure, are both systems.

[0129] • Examples of combinations This technology system can also be configured as follows.

[0130] (1) An information processing method, For a display showing an image as the background of an object, make it display an image in which the amount of light in the area where the object overlaps on the photographic image of a camera covering at least a portion of the display surface of the preceding object has been reduced. (2) As described in the information processing method in note (1), among which, The image of the foreground object, in which the amount of light in the area of ​​the foreground object has been reduced, is displayed in the photograph taken by the foreground camera. (3) As described in the information processing methods in (1) or (2) above, among which, At least a portion of the area where the aforementioned objects overlap is determined as the area where the light intensity is to be reduced, i.e., the light intensity reduction area; Generate a previous display image in which the light intensity of the area where the light intensity has been reduced. (4) As described in the information processing method in note (3), among which, The location, extent, and brightness of the area where the light intensity is reduced are determined based on specified information generated by operating the operation screen. (5) As described in the information processing method in note (4), among which, The specified information is generated by performing specified operations on the specified operation screen based on the image of the foreground as the background of the object. (6) As described in any of the information processing methods in items (3) to (5) above, among which, An image of the area where the light intensity of the preceding image is reduced is set to black and generated as the preceding image display image. (7) As described in any of the information processing methods in items (3) to (5) above, among which, An image that reduces the brightness of the area where the light intensity of the preceding image is reduced is generated as the preceding display image. (8) As described in any of the information processing methods in items (3) to (5) above, among which, An image from which brightness information has been removed from the pixel information of each pixel constituting the area of ​​reduced light intensity is generated as the image of the previous display. (9) As described in any of the information processing methods in items (1) to (8) above, among which, Identify the previously mentioned objects. (10) As described in the information processing method in note (9), among which, Identifying objects based on photographic images captured by a camera. (11) As described in the information processing method in note (9), among which, The object is identified based on the measurement results of a plurality of sensors arranged on the display surface of the front display. (12) As described in the information processing method in note (9), among which, Identify objects based on images captured by a different camera than the one used to capture the original footage. (13) As described in any of the information processing methods in items (9) to (12) above, among which, Based on the identification results of the previously recorded object, at least one of the position, range, and brightness of the area where the previously recorded light intensity is reduced is changed based on the specified information generated by the operation of the operation screen. (14) As described in any of the information processing methods in items (3) to (8) above, among which, The amount of light in the field is adjusted according to the positional relationship between the object in front and the display screen. (15) As described in any of the information processing methods in items (3) to (8) above, among which, Adjust the amount of light in the foreground to reduce the amount of light in the background, depending on the shooting scene. (16) As described in any of the information processing methods in items (3) to (8) above, among which, Adjust the amount of light in the front camera to reduce the amount of light in the field of view, depending on the focus position of the front camera. (17) As described in any of the information processing methods in items (3) to (8) above, among which, The amount of light in the field is adjusted according to the intensity of the light illuminating the object in front, thereby reducing the amount of light in the field. (18) As described in any of the information processing methods in items (3) to (8) above, among which, The position and range of the area where the light intensity of the preceding object is reduced shall change in accordance with the movement of the preceding object. (19) An information processing system, comprising: The display control unit is a display that displays an image of an object as a background, and displays a display image in which the light intensity of the area overlaid by the object on the photographic image of a camera covering at least a portion of the display surface of the pre-capture display has been reduced. (20) A program that instructs a computer to perform the following processes: For a display showing an image as the background of an object, make it display an image in which the amount of light in the area where the object overlaps on the photographic image of a camera covering at least a portion of the display surface of the preceding object has been reduced.

[0131] 1: Camera System 11:PC 12: LED Display 13: Camera for photography H: Actor 14: Lighting machines 15: Lighting machines A1: Areas with reduced light intensity 31: CPU 32:ROM 33: RAM 34: Busbar 35: Input / Output Interface 36: Input Section 37: Output Section 38: Memory Department 39: Communications Department 40: Drive unit 41: Removable Media 51: Information Processing Department 61:Image acquisition department 62: Identification Department 63: Area of ​​Light Reduction Decision Department 64: Display Image Generation Unit 65: Display Control Unit O: Object 81-1, 81-2: Identification Camera P1: Location P2: Location P3: Location E1: Domain Designation Department E2: Brightness specification unit E3: Object Detection Display Unit S: Display screen A1-1: Area of ​​Reduced Light Intensity A1-2: Areas with reduced light intensity 91-1~91-n: Display Units 101-1~101-m: LED array 111: Controller for LED Displays 121: Drive Control Unit 122: LED Block 131: Signal Input IF 132: Signal Processing Department 133-1~133-m: Signal output IF 141-1~141-m: LED driver

Claims

1. An information processing method, comprising, for a display showing an image as a background of an object, causing the display to display a photographic image of a camera covering at least a portion of the display surface of the prior object, in which the light intensity of the area overlaid by the prior object has been reduced; determining at least a portion of the area overlaid by the prior object as the area to be reduced in light intensity, i.e., the light intensity reduction area; and generating a prior display image in which the light intensity of the prior light intensity reduction area has been reduced.

2. The information processing method as described in Request 1, wherein, The image of the foreground object, in which the amount of light in the area of ​​the foreground object has been reduced, is displayed in the photograph taken by the foreground camera.

3. The information processing method as described in Request 1, wherein, The location, extent, and brightness of the area where the light intensity is reduced are determined based on specified information generated by operating the operation screen.

4. The information processing method as described in request item 3, wherein, The specified information in the foreword is generated by performing specified operations on the foreword operation screen based on the image of the foreword as the background of the object.

5. The information processing method as described in Request 1, wherein, An image of the area where the light intensity of the preceding image is reduced is set to black and generated as the preceding image display image.

6. The information processing method as described in Request 1, wherein, An image that reduces the brightness of the area where the front light intensity is reduced is generated as a front display image.

7. The information processing method as described in Request 1, wherein, An image from which brightness information has been removed from the pixel information of each pixel constituting the area of ​​reduced light intensity is generated as the image for display.

8. The information processing method as described in Request 1, wherein, Identify the previously mentioned objects.

9. The information processing method as described in request item 8, wherein, Identifying objects based on photographic images captured by a camera.

10. The information processing method as described in request item 8, wherein, The object is identified based on the measurement results of a plurality of sensors arranged on the display surface of the front display.

11. The information processing method as described in request item 8, wherein, Identify objects based on images captured by a different camera than the one used to capture the original footage.

12. The information processing method as described in request item 8, wherein, Based on the identification results of the previously recorded object, at least one of the position, range, and brightness of the area where the previously recorded light intensity is reduced is changed based on specified information generated by operating the operation screen.

13. The information processing method as described in Request 1, wherein, The amount of light in the field of the front light is adjusted according to the positional relationship between the front object and the front display.

14. The information processing method as described in Request 1, wherein, Adjust the amount of light in the foreground to reduce the amount of light in the surrounding area, depending on the shooting scene.

15. The information processing method as described in Request 1, wherein, Adjust the amount of light in the front camera to reduce the amount of light in the field of view, depending on the focus position of the front camera.

16. The information processing method as described in Request 1, wherein, Adjust the amount of light in the field of illumination according to the intensity of the light illuminating the object in front.

17. The information processing method as described in Request 1, wherein, The position and range of the area where the light intensity of the preceding object is reduced shall change in accordance with the movement of the preceding object.

18. An information processing system comprising: a display control unit, which, for a display showing an image of an object as a background, displays a display image in which the light intensity of the area overlaid by the object on the photographic image of a camera covering at least a portion of the display surface of the display showing the object has been reduced; and determines at least a portion of the area overlaid by the object as the area to be reduced in light intensity, i.e., the light intensity reduction area.

19. An information processing program for instructing a computer to perform the following processing: for a display showing an image as a background of an object, displaying a display image in which the light intensity of the area overlaid by the object on the photographic image of a camera covering at least a portion of the display surface of the display is reduced; determining at least a portion of the area overlaid by the object as the area to be reduced in light intensity, i.e., the light intensity reduction area.

Images