Information processing system and information processing method

Abstract

The present technology relates to an information processing system and an information processing method with which it is possible to prevent the occurrence of a black belt when shooting a display unit. The control unit of a vertical blanking period control unit controls the vertical blanking period of a video displayed in a video wall, on the basis of the exposure time of a video camera that shoots the video wall. The present technology can be applied, for example, to a re-shooting system that is used in virtual production in which re-shooting is conducted against the video displayed on an LED display as a background, or a PC, a video wall controller, a video wall, and a video camera that constitute the re-shooting system.

Description

[Technical field]

[0001] The present technology relates to an information processing system and an information processing method, and more particularly to an information processing system and an information processing method capable of preventing the occurrence of black bands when photographing a display unit. [Background technology]

[0002] In recent years, the market for direct-view displays such as LED (Light Emitting Diode) displays has been expanding. For example, virtual production, in which a background such as a landscape is reproduced on an LED display at a content production site and then the background and subject are photographed (re-photographed) with a camera, is growing rapidly.

[0003] In this type of virtual production, when re-shooting is performed with a rolling shutter image sensor in a camera, black bands, which are stripes of uneven brightness, may appear in the re-shooting image due to a mismatch between the camera's exposure time and the LED display's light emission cycle. Black bands are stripes with lower brightness than other areas, and the color of the black bands varies in shade of black.

[0004] In order to reduce the visibility of the black bands, it has been proposed to shorten the light emission cycle of the LED display, that is, to increase the number of times the LED emits light within an exposure period (see, for example, Patent Document 1).

[0005] However, although the invention described in Patent Document 1 can reduce the visibility of the black belts, it is difficult to prevent the black belts from appearing in the first place.

[0006] Furthermore, black bars may also occur when the number of vertical blanking periods in the image displayed on the LED display, which is included in the camera's exposure period, differs from row to row. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] International Publication No. 2018 / 164105 Summary of the Invention [Problem to be solved by the invention]

[0008] For these reasons, there is a demand for preventing the occurrence of black bands when photographing a display unit, but at present such demand has not been adequately met.

[0009] The present technology has been made in consideration of such circumstances, and makes it possible to prevent the occurrence of black bands when photographing a display unit. [Means for solving the problem]

[0012] One of the technologies The side information processing system is Rolling shutter system an acquisition unit that acquires an exposure time of an image capturing unit that captures an image of a display unit; and a control unit that controls a vertical blanking period of an image displayed on the display unit that emits light by a passive matrix driving method based on the exposure time; , the display unit; wherein the control unit is configured to control the vertical blanking period so that the exposure time is an integer multiple of a light emission period of the display unit. This technology one In one aspect, an information processing method includes: Rolling shutter system The information processing method includes a control step of controlling a vertical blanking period of an image displayed on a display unit that emits light using a passive matrix driving method, based on an exposure time of an imaging unit that images the display unit, and in the control step, the vertical blanking period is controlled so that the exposure time is an integer multiple of the light emission period of the display unit. This technology one In terms of the aspect, Rolling shutter systemA vertical blanking period of an image displayed on a display unit that emits light by a passive matrix driving method is controlled based on an exposure time of an image capturing unit that captures a display unit. Specifically, the vertical blanking period is controlled so that the exposure time is an integer multiple of the light emission period of the display unit.

[0013] The information processing system may be an independent device, a module incorporated in another device, or may be composed of multiple devices.

[0014] The information processing system according to one aspect of the present technology can be realized by causing a computer to execute a program.

[0015] Furthermore, in order to realize the information processing system according to one aspect of the present technology, a program executed by a computer can be provided by transmitting it via a transmission medium or by recording it on a recording medium. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram illustrating an overview of a first embodiment of a re-imaging system to which the present technology is applied. [Diagram 2] FIG. 2 is a diagram showing an image re-captured by the video camera of FIG. 1. [Diagram 3] FIG. 2 is a diagram illustrating an example of the configuration of the re-imaging system of FIG. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of the video camera shown in FIG. [Diagram 5] FIG. 4 is a block diagram showing an example of the hardware configuration of the PC in FIG. [Figure 6] 6 is a block diagram showing an example of a functional configuration of a CPU in FIG. 5. [Figure 7] FIG. 2 is a block diagram showing a configuration example of a video wall controller. [Figure 8] FIG. 2 is a block diagram showing a detailed configuration example of a display unit. [Figure 9] FIG. 1 is a diagram illustrating a configuration example of an LED array. [Figure 10] 1A and 1B are diagrams illustrating the light-emitting operation of an LED array. [Figure 11] FIG. 2 is a diagram illustrating the light emission period of an LED array. [Figure 12] FIG. 1 is a diagram illustrating a cause of black belts. [Figure 13] FIG. 13 is a diagram showing an example of a re-captured image in which black bands appear. [Figure 14] FIG. 4 is a diagram illustrating control of a refresh rate by a control unit. [Figure 15] 5 is a flowchart illustrating a shooting process of the video camera of FIG. 4. [Figure 16] 11 is a flowchart illustrating a refresh rate control process. [Figure 17] 4 is a flowchart illustrating video signal processing. [Figure 18] 11 is a flowchart illustrating a display control signal generation process. [Figure 19] 11 is a flowchart illustrating a display process. [Figure 20] 1A and 1B are diagrams illustrating an example of a re-imaging system that reduces the visibility of black belts using a conventional method. [Figure 21] FIG. 13 is a diagram showing an example of a re-captured image in which the visibility of black bands has decreased. [Figure 22] FIG. 11 is a diagram illustrating a configuration example of a second embodiment of a re-imaging system to which the present technology is applied. [Figure 23] 4 is a block diagram showing an example of the configuration of a vertical blanking interval control unit; [Figure 24] 23 is a block diagram showing a detailed configuration example of the display unit of FIG. 22. [Diagram 25] FIG. 2 is a diagram illustrating a vertical blanking period. [Figure 26] FIG. 1 is a diagram illustrating a cause of black belts. [Figure 27] FIG. 23 is a diagram for explaining the control by the PC in FIG. 22. [Figure 28] 11 is a flowchart illustrating a vertical blanking period control process. [Figure 29]FIG. 13 is a diagram illustrating a configuration example of a third embodiment of a re-imaging system to which the present technology is applied. [Diagram 30] 4 is a block diagram showing an example of the configuration of a vertical blanking interval control unit; [Diagram 31] FIG. 30 is a block diagram showing a detailed configuration example of the display unit of FIG. 29. [Diagram 32] FIG. 2 is a diagram illustrating a vertical blanking period. [Diagram 33] FIG. 4 is a diagram illustrating an exposure time and a vertical blanking period. [Diagram 34] FIG. 1 is a diagram illustrating a cause of black belts. [Diagram 35] FIG. 4 is a diagram illustrating a control mode. [Diagram 36] FIG. 4 is a diagram illustrating a control mode. [Figure 37] 11 is a flowchart illustrating a vertical blanking period control process. [Figure 38] FIG. 11 is a diagram illustrating a refresh rate control process. [Figure 39] FIG. 11 is a diagram illustrating a refresh rate control process. [Diagram 40] FIG. 4 is a diagram illustrating control of a vertical blanking period. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, modes for carrying out the present technology (hereinafter, referred to as embodiments) will be described in the following order. 1. First embodiment (example of configuration of re-imaging system that controls refresh rate) 2. Second embodiment (Example of configuration of a re-imaging system that also controls the vertical blanking period) 3. Third embodiment (example of configuration of re-imaging system considering the case where the exposure period includes a vertical blanking period)

[0018] In the drawings referred to in the following description, the same or similar parts are denoted by the same or similar reference numerals.

[0019] <First embodiment> <Overview of the re-imaging system> FIG. 1 is a diagram illustrating an overview of a first embodiment of a re-imaging system including an information processing system to which the present technology is applied.

[0020] 1, the re-imaging system 10 is a system used, for example, in virtual production, and includes a video camera 11 and a video wall 12. A photographer 21 uses the video camera 11 to re-imaging a subject, a motorcycle 24, with an image 23 displayed on a video wall 12 installed in a studio 22 as the background.

[0021] <Example of re-shot footage> FIG. 2 shows an image re-captured by the video camera 11 of FIG.

[0022] 2, the image 30 re-shot by the video camera 11 is an image in which the motorcycle 24 appears to be in the location where the image 23 was shot. In this way, by using the re-shooting system 10 to perform re-shooting, the photographer 21 can shoot the image 30 in the studio 22 in which the motorcycle 24 appears to be in the location where the image 23 was shot.

[0023] <Example of re-imaging system configuration> FIG. 3 is a diagram showing an example of the configuration of the re-imaging system 10. As shown in FIG.

[0024] As shown in FIG. 3, the re-imaging system 10 (information processing system) is configured with a video camera 11 and a video wall 12, as well as a video server 61, a PC (personal computer) 62, and a video wall controller 63.

[0025] Video camera 11 is a photographing device that photographs using a rolling shutter method. Video camera 11 accepts input from photographer 21 and sets the exposure time (opening angle) and the like. In response to a shooting start instruction from photographer 21, video camera 11 photographs the image displayed on video wall 12 with the set exposure time, with the image as the background. Note that multiple video cameras similar to video camera 11 may be used in re-imaging system 10.

[0026] The video wall 12 (display section) is a large LED display in which n (n is a positive integer) display units (cabinets) 41-1 to 41-n are arranged in a tiled pattern. Fig. 3 illustrates a case where n is 6 x 24. In the following, when there is no need to distinguish between the display units 41-1 to 41-n, they will simply be referred to as display units 41.

[0027] The display unit 41 is configured by arranging m (m is a positive integer) LED arrays 51-1 to 51-m in a tiled shape, with LEDs (not shown) corresponding to each pixel being arranged in a matrix (two-dimensional array). Fig. 3 illustrates a case where m is 4 x 3. In the following, when it is not necessary to distinguish between the LED arrays 51-1 to 51-m, they will simply be referred to as the LED array 51.

[0028] Each display unit 41 of the video wall 12 is connected to the video wall controller 63. Based on a frame-by-frame video signal supplied from the video wall controller 63 and a display control signal that controls the display of the video wall 12, the video wall 12 displays a video corresponding to the video signal in frame units. Specifically, the video signal supplied to each display unit 41 is a video signal that corresponds to the position of that display unit 41 on the video wall 12. Each display unit 41 displays a video based on that video signal in accordance with the display control signal, thereby displaying one frame of video on the entire video wall 12.

[0029] The video server 61 is, for example, a server computer, and supplies frame-by-frame video signals of video content and the like to the video wall controller 63. Instead of the video server 61, frame-by-frame video signals may be provided to the video wall controller 63 from a recording medium such as a PC or a Blu-ray Disc (BD) (registered trademark).

[0030] The PC 62 (information processing system) is a general-purpose computer. The PC 62 generates control commands for controlling the video wall controller 63 and transmits them to the video wall controller 63, thereby controlling the video wall controller 63.

[0031] For example, PC 62 acquires an exposure time transmitted from video camera 11. PC 62 determines a refresh rate of video wall 12 so that the exposure time corresponds to the light emission cycle of the LEDs of video wall 12. The refresh rate is the number of times the LEDs emit light per second. PC 62 controls the refresh rate by transmitting a refresh rate command, which is a control command representing the determined refresh rate, to video wall controller 63.

[0032] The video wall controller 63 performs predetermined signal processing on the frame-by-frame video signal supplied from the video server 61. The video wall controller 63 divides the resulting frame-by-frame video signal into n parts according to the position of each display unit 41, and transmits them to each display unit 41. Based on a refresh rate command supplied from the PC 62, the video wall controller 63 also transmits to each display unit 41 a display control signal indicating the refresh rate, etc., indicated by the refresh rate command.

[0033] In the re-imaging system 10 configured as described above, each device is frame-synchronized by a genlock signal. Therefore, for example, the exposure start time of each frame in the video camera 11 and the display start time of each frame on the video wall 12 are synchronized.

[0034] The video wall controller 63 and the video wall 12 may be integrated into one unit, or may be a display device in which they are integrated. The PC 62, the video wall controller 63, and the video wall 12 may be integrated into one unit, or may be a display device in which they are integrated.

[0035] <Example of video camera configuration> FIG. 4 is a block diagram showing an example of the configuration of the video camera 11 of FIG.

[0036] The video camera 11 in FIG. 4 is made up of an input section 81, a setting section 82, a driving section 83, an optical system 84, a shooting section 85, a signal processing section 86, a monitor 87, a video output section 88, and a memory 89.

[0037] The input unit 81 receives an input from the photographer 21 and generates an input signal representing the input content. For example, the photographer 21 operates the input unit 81 to input an appropriate exposure time, taking into consideration the constraints of the dynamic range when converting (the intensity of) light into (the amount of) signal charge in the imaging unit 85. In this case, the input unit 81 generates an input signal representing the exposure time and supplies it to the setting unit 82. When the photographer 21 operates the input unit 81 to input an instruction to start imaging, the input unit 81 generates an input signal representing the start of imaging and supplies it to the drive unit 83.

[0038] The setting unit 82 sets shooting information, which is various information related to shooting such as the exposure time represented by the input signal from the input unit 81, based on the input signal from the input unit 81. The setting unit 82 supplies the shooting information to the drive unit 83, and transmits the exposure time, which is included in the shooting information, to the PC 62 via a wire such as a coaxial cable.

[0039] The drive unit 83 generates a drive signal, which is a control signal for driving each pixel unit arranged in a matrix on the light-receiving surface of the imaging unit 85, based on the imaging information from the setting unit 82, the imaging start instruction from the input unit 81, etc., and outputs it to the imaging unit 85.

[0040] The optical system 84 is configured to have one or a plurality of lenses, guides light (incident light) from a subject or the like to the imaging unit 85, and forms an image on the light-receiving surface of the imaging unit 85.

[0041] The imaging unit 85 is configured by a CMOS (Complementary Metal Oxide Semiconductor) image sensor or the like. The imaging unit 85 drives each pixel unit according to the drive signal supplied from the drive unit 83 and performs imaging in a rolling shutter method. Specifically, the imaging unit 85 outputs an electrical signal corresponding to the charge accumulated in each pixel unit in row units according to the drive signal supplied from the drive unit 83. Then, the imaging unit 85 starts accumulating charges corresponding to the light incident on each pixel unit via the optical system 84 in row units. The imaging unit 85 reads out an electrical signal corresponding to the charge accumulated for the exposure time set by the setting unit 82 in row units according to the drive signal supplied from the drive unit 83, and transfers it to the signal processing unit 86 in column units. As described above, the imaging unit 85 performs imaging for the exposure time set by the setting unit 82.

[0042] The signal processing unit 86 performs various signal processes on the electrical signal corresponding to the charge transferred from the imaging unit 85, and converts it into a video signal in frame units of a predetermined digital video format. This video signal in frame units is supplied to the monitor 87 or the video output unit 88, or is supplied to the memory 89 and stored (recorded). The monitor 87 displays the video in frame units based on the video signal in frame units supplied from the signal processing unit 86. The video output unit 88 outputs the video signal in frame units supplied from the signal processing unit 86 to the outside of the video camera 11.

[0043] <Configuration example of PC> FIG. 5 is a block diagram showing an example of the hardware configuration of the PC 62 in FIG.

[0044] In the PC 62 of FIG. 5, a central processing unit (CPU) 101, a read only memory (ROM) 102, and a random access memory (RAM) 103 are interconnected via a bus 104.

[0045] An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

[0046] The input unit 106 includes a keyboard, a mouse, a microphone, etc. The output unit 107 includes a display, a speaker, etc. The storage unit 108 includes a hard disk, a non-volatile memory, etc. The communication unit 109 includes a network interface, etc. The drive 110 drives removable media 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0047] In the PC 62 configured as above, the CPU 101 performs various processes by, for example, loading a program stored in the storage unit 108 into the RAM 103 via the input / output interface 105 and the bus 104 and executing the program. For example, the CPU 101 performs refresh rate control process of generating a refresh rate command and controlling the communication unit 109 to transmit the refresh rate to the video wall controller 63.

[0048] The program executed by the PC 62 (CPU 101) can be provided by being recorded on a removable medium 111 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0049] In the PC 62, the program can be installed in the storage unit 108 via the input / output interface 105 by mounting the removable medium 111 in the drive 110. The program can also be received by the communication unit 109 via a wired or wireless transmission medium and installed in the storage unit 108. Alternatively, the program can be pre-installed in the ROM 102 or the storage unit 108.

[0050] The programs executed by PC 62 may be programs that are processed chronologically in the order described in this specification, or may be programs that are processed in parallel or at the required timing, such as when called.

[0051] <Configuration example of refresh rate control unit> FIG. 6 is a block diagram showing an example of a functional configuration of the CPU 101 in FIG. 5 when the CPU 101 performs the refresh rate control process.

[0052] When performing the refresh rate control process, the CPU 101 functions as the refresh rate control unit 130 in Fig. 6. The refresh rate control unit 130 is made up of an acquisition unit 131, a control unit 132, and a transmission control unit 133.

[0053] The acquisition unit 131 acquires the exposure time transmitted from the setting unit 82 of the video camera 11 and received via the communication unit 109. The acquisition unit 131 supplies the exposure time to the control unit 132.

[0054] The control unit 132 determines the refresh rate of the video wall 12 so that the exposure time supplied from the acquisition unit 131 corresponds to the light emission cycle of the LEDs of the video wall 12. The control unit 132 controls the refresh rate by generating a refresh rate command, which is a control command (control signal) that indicates the refresh rate. The control unit 132 outputs the refresh rate command to the transmission control unit 133.

[0055] The transmission control unit 133 supplies the refresh rate command supplied from the control unit 132 to the communication unit 109, and instructs the communication unit 109 to transmit the refresh rate command to the video wall controller 63. As a result, the communication unit 109 transmits (outputs) the refresh rate command to the video wall controller 63 via a wired line such as a LAN (Local Area Network) cable.

[0056] <Video wall controller configuration example> FIG. 7 is a block diagram showing an example of the configuration of the video wall controller 63 of FIG.

[0057] The video wall controller 63 is composed of a LAN terminal 151, an HDMI (High Definition Multimedia Interface) (registered trademark) terminal 152, a DP (Display Port) terminal 153, a DVI (Digital Visual Interface) terminal 154, a network IF (Interface) 155, an MPU (Micro Processor Unit) 156, a signal input IF 157, a signal processing unit 158, a DRAM (Dynamic Random Access Memory) 159, a distribution unit 160, and signal output IFs 161-1 to 161-n.

[0058] The LAN terminal 151 is a connection terminal for a LAN cable, and is connected to a LAN cable that is connected to the communication unit 109 of the PC 62. The LAN terminal 151 receives control commands such as a refresh rate command via the LAN cable, and supplies them to the MPU 156 via the network IF 155.

[0059] The MPU 156 generates a display control signal in accordance with a refresh rate command supplied via the network IF 155. The MPU 156 also generates a video adjustment signal that instructs adjustment of the video signal in accordance with a video adjustment command, which is a control command related to adjustment of the video signal, supplied via the network IF 155. The MPU 156 supplies the display control signal and the adjustment signal to the signal processor 158.

[0060] 7, a video server 61 and the HDMI terminal 152 are connected to each other, and a frame-by-frame video signal supplied from the video server 61 is input to the HDMI terminal 152 as an input unit and acquired by the HDMI terminal 152. The frame-by-frame video signal input to the HDMI terminal 152 is supplied to a signal input IF 157.

[0061] In the example of Figure 7, the video server 61 is connected to the HDMI terminal 152, but the HDMI terminal 152, DP terminal 153, and DVI terminal 154 all have basically the same functions and only differ in standard, so any one of them is selected and connected as necessary.

[0062] The signal input IF 157 converts the frame-by-frame video signal supplied from the HDMI terminal 152 into a frame-by-frame video signal in a predetermined video format, and supplies it to a signal processing unit 158 ​​.

[0063] The signal processing unit 158 ​​performs necessary signal processing for the entire video wall 12 on the frame-by-frame video signal supplied via the signal input IF 157 based on the video adjustment signal supplied from the MPU 156, while exchanging data with the DRAM 159 as necessary. Specifically, the signal processing unit 158 ​​adjusts the color temperature, contrast, brightness, etc. of the video signal based on the video adjustment signal. The signal processing unit 158 ​​supplies the adjusted frame-by-frame video signal and the display control signal supplied from the MPU 156 to the distribution unit 160.

[0064] The distribution unit 160 distributes the frame-by-frame video signal supplied from the signal processing unit 158 ​​to the signal output IFs 161-1 to 161-n connected to the display units 41-1 to 41-n, respectively. Specifically, the distribution unit 160 (splitting unit) splits the frame-by-frame video signal into n parts according to the position of each display unit 41, and supplies each of the signal output IFs 161-1 to 161-n connected to the display unit 41. In the following, when it is not necessary to individually distinguish the signal output IFs 161-1 to 161-n, they will simply be referred to as signal output IFs 161. The distribution unit 160 supplies the display control signal supplied from the signal processing unit 158 ​​to each signal output IF 161.

[0065] The signal output IF 161 (output unit) transmits the frame-by-frame video signal and the display control signal supplied from the distribution unit 160 to the display unit 41 connected to itself. The transmission medium for this transmission may be a wired medium such as a LAN cable, or may be wireless.

[0066] In this embodiment, the control command is automatically transmitted from the PC 62, but it may be transmitted in response to a request from the video wall controller 63.

[0067] <Example of detailed display unit configuration> FIG. 8 is a block diagram showing an example of a detailed configuration of the display unit 41. As shown in FIG.

[0068] As shown in FIG. 8, the display unit 41 includes a driver control unit 191 and an LED block 192.

[0069] The driver control unit 191 includes a signal input IF 211, a signal processing unit 212, a DRAM 213, and signal output IFs 214-1 through 214-m.

[0070] The signal input IF 211 receives a video signal and a display control signal in frame units transmitted from the video wall controller 63 , and supplies them to a signal processing unit 212 .

[0071] The signal processing unit 212 performs signal processing required for each display unit 41 on the frame-by-frame video signal supplied from the signal input IF 211 while exchanging data with the DRAM 213 as necessary. Specifically, the signal processing unit 212 performs color and brightness correction required for each display unit 41 on the frame-by-frame video signal using a color matrix LUT (Look Up Table) stored in the DRAM 213 or the like. The signal processing unit 212 divides the frame-by-frame video signal after signal processing into m pieces according to the position of each LED array 51 on the display unit 41. For each LED array 51, the signal processing unit 212 generates a display signal for setting the light emission intensity of each LED constituting the LED array 51 on a frame-by-frame basis based on the frame-by-frame video signal after signal processing.

[0072] The signal processing unit 212 is connected to signal output IFs 214-1 to 214-m corresponding to the LED arrays 51-1 to 51-m, respectively. In the following, when there is no need to distinguish between the signal output IFs 214-1 to 214-m, they will simply be referred to as signal output IFs 214. The signal processing unit 212 supplies a display signal of each LED array 51 to the signal output IF 214 corresponding to that LED array 51. The signal processing unit 212 also supplies a display control signal, supplied from the signal input IF 211, to each signal output IF 214.

[0073] The signal output IF 214 transmits the display signal and the display control signal supplied from the signal processing unit 212 to the LED block 192 .

[0074] The LED block 192 is made up of LED drivers 221-1 to 221-m and LED arrays 51-1 to 51-m.

[0075] The LED drivers 221-1 to 221-m are respectively connected to the signal output IFs 214-1 to 214-m of the driver control unit 191 and are also respectively connected to the LED arrays 51-1 to 51-m. In the following, when there is no need to distinguish the LED drivers 221-1 to 221-m individually, they are simply referred to as the LED driver 221.

[0076] Based on the display control signal and the display signal transmitted from the signal output IF connected to it, the LED driver 221 drives each LED by performing PWM (Pulse Width Modulation) control on the light emission of each LED constituting the LED array 51 connected to it. Specifically, the LED driver 221 generates a PWM signal, which is a digital signal, so that each LED 241 of the LED array 51 emits light at the intensity based on the display signal corresponding to that LED 241 for the refresh rate represented by the display control signal per second, and supplies it to each LED 241. As a result, on the LED array 51, an image corresponding to the position on the video wall 12 of that LED array 51 among the images corresponding to the frame unit video signals output from the video server 61 is displayed.

[0077] <Configuration Example of LED Array> FIG. 9 is a diagram showing a configuration example of the LED array 51 in FIG. 8.

[0078] As shown in FIG. 9, the light emission of the LED array 51 is controlled by a passive matrix driving method.

[0079] Specifically, the LED array 51 is composed of p×p LEDs 241 (p is a positive number), as well as luminance control wirings (Sig line) 251 and row selection wirings (Scan line) 252. The LED 241 is a Common Cathode type LED. The p×p LEDs 241 are arranged in a matrix with p in the row direction (vertical direction) and p in the column direction (horizontal direction). In the example of FIG. 9, the number of LEDs 241 in the row direction and the column direction is the same p, but the number of LEDs 241 in the row direction and the column direction may be different.

[0080] The brightness control wiring 251 is provided for each column of the LEDs 241, and the same brightness control wiring 251 is connected to the LEDs 241 in each column. The row selection wiring 252 is provided for each row of the LEDs 241, and the same row selection wiring 252 is connected to the LEDs 241 in each row.

[0081] Based on the display control signal supplied from the signal output IF 214, the LED driver 221 controls the light emission timing of the LEDs 241 in each row so that the LEDs 241 emit light in a light emission cycle corresponding to the refresh rate represented by the display control signal, row by row.

[0082] Specifically, the LED driver 221 adjusts at least one of the frequency of the reference clock and the interval until the next row emits light so that the LEDs 241 emit light row by row in order from the upper row, and the light emission is performed at intervals corresponding to the refresh rate represented by the display control signal. Thereby, the LED driver 221 applies a predetermined fixed potential to the row selection wiring 252 corresponding to that row at the light emission timing of the LEDs 241 in each row. Although the GND potential (0V potential) is generally used as the predetermined fixed potential, this is not the only case.

[0083] The LED driver 221 also inputs the PWM signal of each LED 241 in that row to the brightness control wiring 251 corresponding to the column of that LED 241 at the light emission timing of the LEDs 241 in each row, based on the frame unit display signal supplied from the signal output IF 214.

[0084] As described above, in the LED array 51, at the light emission timing of the LEDs 241 in each row, each LED 241 in that row emits light of a brightness based on the display signal of that LED 241. As a result, in the LED array 51, an image corresponding to the video signal is displayed row by row in order from the upper row, and this image is refreshed (updated) at the refresh rate per second.

[0085] <Explanation of the light emission operation of the LED array> FIG. 10 is a diagram for explaining the light emitting operation of the LED array 51. As shown in FIG.

[0086] In the example of FIG. 10, the LED array 51 is made up of 15×15 LEDs 241 .

[0087] As shown in A of FIG. 10, first, in the LED array 51, the LEDs 241 in the first row from the top emit light at a luminance corresponding to the display signal under the control of the LED driver 221. Next, as shown in B of FIG. 10, the LEDs 241 in the second row from the top emit light at a luminance corresponding to the display signal under the control of the LED driver 221. Similarly, the rows of the LEDs 241 to be illuminated are successively lowered, and as shown in C of FIG. 10, the LEDs 241 in the bottom row emit light at a luminance corresponding to the display signal under the control of the LED driver 221. In this manner, an image corresponding to the position of the LED array 51 on the video wall 12 among images corresponding to one frame of video signals is displayed on the LED array 51. In this specification, this is referred to as one scan.

[0088] After the first scan, under the control of the LED driver 221, the LEDs 241 in the first row from the top emit light again at a luminance corresponding to the display signal, as shown in A of FIG. 10. Thereafter, the emission of the LEDs 241 in rows is repeated in a similar manner, and scanning is performed at the refresh rate per second. For example, if the refresh rate represented by the display control signal is 3840 Hz and the frame rate of the video signal is 60p (60 frames / second in the progressive format, i.e., 60 Hz), the number of scans within one frame period is 64 (=3840 Hz / 60 Hz). The refresh rate is the number of scans performed per second, and can therefore also be called the scan speed.

[0089] <Explanation of the light emission cycle> FIG. 11 is a diagram illustrating the light emission period of the LED array 51. As shown in FIG.

[0090] In the example of Fig. 11, the LED array 51 is composed of 15 x 15 LEDs 241. In the graph of Fig. 11, the horizontal axis represents the elapsed time from when display on the video wall 12 starts, and the vertical axis represents the rows of the LED array 51. In the graph of Fig. 11, the i-th row from the top of the LED array 51 is represented as row #i. A rectangle in the graph of Fig. 11 indicates that the LEDs 241 in the row corresponding to the vertical position of the rectangle emit light at a luminance corresponding to the display signal at the elapsed time corresponding to the horizontal position of the rectangle.

[0091] 11, in the LED array 51, the LEDs 241 emit light in a row unit starting from the top row. When all the LEDs 241 in rows #1 to #15 have finished emitting light, that is, when one scan is completed, the next scan is started. The period of one scan is the light emission period of the LEDs 241, which is the reciprocal of the refresh rate. For example, when the refresh rate is 3840 Hz, the light emission period is 1 / 3840 seconds.

[0092] <Explanation of the cause of black belt occurrence> FIG. 12 is a diagram for explaining the cause of black bars appearing in an image re-captured by the re-capture system when the exposure time is not an integer multiple of the LED light emission cycle.

[0093] In FIG. 12, the horizontal axis represents the time elapsed since the start of display, and the vertical axis represents the row of LEDs, with the higher the position in the figure, the higher the position of the row of LEDs. In the example of FIG. 12, the LED array comprises 15 rows of LEDs, and three LED arrays arranged in the row direction (vertical direction) are shown. In FIG. 12, a patterned square indicates that the LEDs in the row corresponding to the vertical position of the square will emit light at the elapsed time corresponding to the horizontal position of the square. The pattern of the square is different for each LED array that includes the row of LEDs corresponding to the square. The same applies to FIG. 14 described later.

[0094] As shown in Figure 12, first, the LEDs in the first row from the top of each LED array light up. Next, the LEDs in the second row from the top of each LED array light up. Similarly, the LEDs in each LED array light up row by row, starting from the top row, until the LEDs in the last row light up, and each LED array ends its first scan. When each LED array finishes its first scan, it starts its second scan.

[0095] Thereafter, each LED array repeats scanning in the same manner, and each LED array scans a number of times within one frame period that corresponds to the refresh rate. In the example of Fig. 12, the refresh rate is four times the frame rate, and scanning is performed four times within one frame period. That is, in the example of Fig. 12, when the frame rate is 60p, the refresh rate is 240Hz.

[0096] On the other hand, the video camera shoots using the rolling shutter method with a specified exposure time. Therefore, exposure to the light from the LEDs of each row is performed in order from the top row, and the exposure start timing of each row is different. Note that each device in the re-shooting system is frame synchronized, so the exposure start time and display start time of one frame are synchronized.

[0097] In such a case, when the exposure time of the video camera does not correspond to the light emission cycle (scan cycle) of the LED, that is, when the exposure time is not an integer multiple of the light emission cycle of the LED, the number of scans (number of times the LED emits light) during the exposure period may not be uniform in each row. In the example of FIG. 12, in rows whose exposure period falls within the range of the rectangle 261, the LED emits light only once during the exposure period, but in rows whose exposure period falls outside the range of the rectangle 261, the LED emits light twice during the exposure period. Therefore, the luminance of pixels in rows whose exposure period falls within the range of the rectangle 261 and are re-captured by the video camera is lower than the luminance of pixels in rows whose exposure period falls outside the range of the rectangle 261. As a result, the images of the rows whose exposure period falls within the range of the rectangle 261 among the images re-captured by the video camera have black bands.

[0098] <Example of re-shot footage with black bars> FIG. 13 is a diagram showing an example of a re-captured image in which black bars occur due to the cause described in FIG.

[0099] In the example of Fig. 13, the refresh rate is 3840 Hz. In the example of Fig. 13, a video camera re-shoots a person holding a table tennis racket as a subject, with an image displayed on a video wall as the background. In the re-shot image 280 obtained as a result, horizontal black stripes 281 may appear as artifacts due to the cause described in Fig. 12. In virtual production, the appearance of black stripes 281 in the re-shot image that did not exist on location reduces the quality of the re-shot image.

[0100] <Explanation of refresh rate control> FIG. 14 is a diagram for explaining the control of the refresh rate by the control unit 132 in FIG.

[0101] 14, the control unit 132 determines the refresh rate so that the exposure time corresponds to the light emission cycle of the video camera 11. Specifically, for example, the control unit 132 synchronizes the exposure start time with the scan start time, and controls the refresh rate so that the exposure time is an integer multiple of the light emission cycle.

[0102] In the example of Fig. 14, the control unit 132 controls the refresh rate to 4.6 times the frame rate so that the exposure time is twice the light emission period. This increases the scanning speed compared to the case of Fig. 12, and all rows are scanned twice within the exposure period. As a result, no rows of pixels with low brightness are generated in the video signal recaptured by the video camera 11 in units of frames, and no black bands are generated.

[0103] In the example of Figure 14, the refresh rate is controlled to be higher than in the example of Figure 12, but the refresh rate may be controlled to be lower than in the example of Figure 12 as long as it is controlled so that the exposure time and the light emission period correspond to each other.

[0104] <Explanation of the shooting process> 15 is a flowchart for explaining the shooting process of the video camera 11. This shooting process is started, for example, when the cameraman 21 operates the input unit 81 to input a shooting start instruction.

[0105] In step S10 of FIG. 15, the input unit 81 receives an input of an instruction to start shooting from the photographer 21, generates an input signal representing the start of shooting, and supplies it to the drive unit 83.

[0106] In step S11, the setting unit 82 transmits the exposure time, which is one of the set shooting information, to the PC 62, and supplies the shooting information to the drive unit 83. This shooting information is input by the photographer 21 operating the input unit 81, and is set by the setting unit 82 based on an input signal generated by the input unit 81 in response to the input.

[0107] In step S12, the driving unit 83 generates a driving signal based on the imaging information supplied from the setting unit 82 in step S11, and supplies the driving signal to the imaging unit 85.

[0108] In step S13, the imaging unit 85 performs imaging using the rolling shutter method in accordance with the drive signal supplied from the drive unit 83 in step S12, and transfers to the signal processing unit 86 the charge accumulated in each pixel unit for the exposure time.

[0109] In step S14, the signal processing unit 86 performs various signal processes on the charge signal transferred from the imaging unit 85, and converts it into a video signal in frame units of a predetermined digital video format.

[0110] In step S15, the signal processing unit 86 supplies the frame-by-frame video signal obtained as a result of the processing in step S14 to the monitor 87, thereby displaying frame-by-frame video corresponding to the video signal. At this time, the signal processing unit 86 also supplies the frame-by-frame video signal obtained as a result of the processing in step S14 to the video output unit 88 as necessary, and outputs it to the outside of the video camera 11.

[0111] In step S16, the signal processing unit 86 supplies the frame-by-frame video signal obtained as a result of the process in step S14 to the memory 89, thereby causing the video signal to be stored in the memory 89.

[0112] In step S17, the input unit 81 determines whether or not it has received an input of an instruction to end shooting from the photographer 21. If it is determined in step S17 that the input of an instruction to end shooting has not yet been received, the process proceeds to step S18.

[0113] In step S18, the setting unit 82 determines whether the exposure time has been changed. Specifically, the setting unit 82 determines whether an input signal representing a new exposure time has been supplied from the input unit 81. If an input signal representing a new exposure time has been supplied from the input unit 81, the setting unit 82 determines that the exposure time has been changed in step S18. Then, the setting unit 82 sets the exposure time as the new exposure time, and returns the process to step S11 to repeat the subsequent processes.

[0114] On the other hand, if an input signal indicating a new exposure time is not provided from the input unit 81, the setting unit 82 determines that the exposure time has not been changed in step S18. Then, the setting unit 82 returns the process to step S13 and repeats the subsequent processes.

[0115] If it is determined in step S17 that the input of the instruction to end shooting has been accepted, the shooting process ends.

[0116] <Explanation of refresh rate control process> Fig. 16 is a flowchart for explaining the refresh rate control process of the refresh rate control unit 130 of Fig. 6. This refresh rate control process is started, for example, when the exposure time is transmitted from the setting unit 82 of the video camera 11 by the process of step S11 of Fig. 15.

[0117] In step S31 of FIG. 16, the acquisition unit 131 acquires the exposure time transmitted from the setting unit 82 and received via the communication unit 109, and supplies the exposure time to the control unit 132.

[0118] In step S32, the control unit 132 determines the refresh rate of the video wall 12 based on the exposure time supplied from the acquisition unit 131 in step S31 so that the exposure time corresponds to the light emission cycle of the LEDs 241. The control unit 132 generates a refresh rate command representing the refresh rate and supplies it to the transmission control unit 133.

[0119] In step S33, the transmission control unit 133 supplies the refresh rate command supplied from the control unit 132 in step S32 to the communication unit 109 and instructs the communication unit 109 to transmit the refresh rate command to the video wall controller 63. Then, the process ends.

[0120] <Explanation of video signal processing> 17 is a flowchart for explaining the video signal processing of the video wall controller 63. This video signal processing starts, for example, when a video signal is transmitted from the video server 61 in frame units.

[0121] In step S51 of FIG. 17, the signal input IF 157 of the video wall controller 63 receives the video signal transmitted from the video server 61 in units of frames via the HDMI terminal 152.

[0122] In step S 52 , the signal input IF 157 converts the frame-by-frame video signal received in step S 51 into a frame-by-frame video signal of a predetermined video format, and supplies it to the signal processing unit 158 ​​.

[0123] In step S53, MPU 156 determines whether or not a video adjustment command has been received from communication unit 109 of PC 62 via LAN terminal 151 and network IF 155. If MPU 156 determines in step S53 that a video adjustment command has not yet been received, it waits until a video adjustment command is received.

[0124] On the other hand, if the MPU 156 determines in step S53 that it has received a video adjustment command, the process proceeds to step S54. In step S54, the MPU 156 generates a video adjustment signal in accordance with the video adjustment command, and supplies the video adjustment signal to the signal processor 158.

[0125] In step S55, the signal processing unit 158 ​​performs necessary signal processing on the frame-by-frame video signal for the entire video wall 12 based on the video adjustment signal while exchanging data with the DRAM 159 as necessary. The signal processing unit 158 ​​supplies the adjusted frame-by-frame video signal to the distribution unit 160.

[0126] In step S56, the distribution unit 160 distributes the frame-by-frame video signal supplied from the signal processing unit 158 ​​in step S55 to each signal output IF 161.

[0127] In step S57, each signal output IF 161 transmits the frame-by-frame video signal supplied from the distribution unit 160 in step S56 to the display unit 41 connected to itself, and then the video signal processing ends.

[0128] <Description of display control signal generation process> 18 is a flowchart for explaining the display control signal generation process of the video wall controller 63. This display control signal generation process is started, for example, when a video signal is transmitted from the video server 61 in frame units.

[0129] 18, the MPU 156 determines whether or not a refresh rate command has been received from the communication unit 109 of the PC 62 via the LAN terminal 151 and the network IF 155. If it is determined in step S71 that a refresh rate command has been received, the MPU 156 advances the process to step S72.

[0130] In step S72, the MPU 156 generates a display control signal in accordance with the received refresh rate command, and supplies the display control signal to the distributor 160 via the signal processor 158. Then, the process proceeds to step S74.

[0131] On the other hand, if it is determined in step S71 that the refresh rate command has not yet been received, in step S73, the MPU 156 generates a display control signal representing a predetermined refresh rate, and supplies the display control signal to the distribution unit 160 via the signal processing unit 158. Then, the process proceeds to step S74.

[0132] In step S74, the distribution unit 160 supplies the display control signal to each signal output IF 161. In step S75, each signal output IF 161 transmits the display control signal supplied from the distribution unit 160 in step S74 to the display unit 41 connected to itself. Then, the display control signal generation process ends.

[0133] <Explanation of display process> Fig. 19 is a flowchart for explaining the display process of each display unit 41. This display process is started, for example, when a video signal is transmitted by the process of step S57 in Fig. 17 and a display control signal is transmitted by the process of step S75 in Fig. 18.

[0134] In step S 91 of FIG. 19, the signal input IF 211 receives a video signal and a display control signal in frame units transmitted from the video wall controller 63 , and supplies them to the signal processing unit 212 .

[0135] In step S92, the signal processing unit 212 performs signal processing required for its own display unit 41 individually on the frame-by-frame video signal supplied from the signal input IF 211 in step S91, while exchanging data with the DRAM 213 as necessary.

[0136] In step S93, the signal processing unit 212 divides the frame-by-frame video signal after signal processing into m parts according to the positions of each LED array 51 on its own display unit 41. In step S94, the signal processing unit 212 generates, for each LED array 51, a display signal for each LED constituting that LED array 51, in frame units, based on the frame-by-frame video signal after signal processing.

[0137] In step S95, the signal processing unit 212 supplies the display signal of each LED array 51 to the signal output IF 214 corresponding to that LED array 51, and also supplies the display control signal supplied from the signal input IF 211 in the processing of step S91 to each signal output IF 214.

[0138] In step S96, each signal output IF 214 transmits a video signal and a display control signal to the LED driver 221 connected to itself.

[0139] In step S97, each LED driver 221 receives a display signal and a display control signal transmitted from the signal output IF 214 connected to itself. In step S98, each LED driver 221 drives each LED 241 by PWM-controlling the light emission of each LED 241 constituting the LED array 51 connected to itself based on the display signal and the display control signal. As a result, each LED array 51 displays an image corresponding to the position of that LED array 51 on the video wall 12, out of images corresponding to the frame-by-frame video signal output from the video server 61.

[0140] As described above, in the re-imaging system 10, the PC 62 controls the refresh rate of the video wall 12 so that the exposure time of the video camera 11 that captures the video wall 12 corresponds to the light emission cycle of the LED 241. Therefore, it is possible to prevent the occurrence of black bars when the video wall 12 is re-imaging.

[0141] In addition, since PC 62 acquires the exposure time and controls the refresh rate based on that exposure time, even if the exposure time changes, the refresh rate can be easily controlled based on the changed exposure time, thereby preventing the occurrence of black bands.

[0142] <An example of a re-imaging system> FIG. 20 is a diagram illustrating an example of a re-imaging system that reduces the visibility of black belts using a conventional method.

[0143] In the example of Fig. 20, the LED array has four rows of LEDs. In the graph of Fig. 20, the horizontal axis represents the time elapsed since the start of display on the video wall, and the vertical axis represents the rows of the LED array. In the graph of Fig. 20, the i-th row from the top of the LED array is represented as row #i. A rectangle in the graph of Fig. 20 indicates that the LEDs in the row corresponding to the vertical position of the rectangle will emit light at the elapsed time corresponding to the horizontal position of the rectangle.

[0144] A conventional re-imaging system that reduces the visibility of black belts by shortening the LED light emission period, i.e., by increasing the refresh rate, reduces the visibility of black belts. Specifically, the re-imaging system doubles the refresh rate shown in the upper part of Fig. 20, for example, as shown in the lower part of Fig. 20.

[0145] This makes it possible to suppress uneven brightness among the rows, even if the exposure time of the video camera does not correspond to the light emission cycle of the LEDs, and the number of times the LEDs emit light within the exposure period in each row is not uniform.

[0146] For example, when the number of scans in one frame period is 64, if the difference between the number of times a row emits light and other rows is -1, the luminance of that row is reduced by 1 / 64 (approximately 1.56%) compared to the luminance of the other rows. In contrast, when the refresh rate is doubled and the number of scans in one frame period is 128, if the difference between the number of times a row emits light and other rows is -1, the luminance of that row is reduced by only 1 / 128 (approximately 0.78%) compared to the luminance of the other rows. As described above, by increasing the refresh rate, the non-uniformity of luminance in each row is suppressed.

[0147] FIG. 21 is a diagram showing an example of a re-captured image in which the visibility of black bands has been reduced by the re-capture system described with reference to FIG.

[0148] In the example of FIG. 21, the refresh rate is 7680 Hz, and the same subject and background as in FIG. 13 are re-captured.

[0149] As shown in FIG. 21, horizontally striped black bands 301 also occur in the re-shot image 300, which has a refresh rate of 7680 Hz, twice that of 3840 Hz. However, the black bands 301 are less visible than the black bands 281 in the re-shot image 280 of FIG. 13, which has a refresh rate of 3840 Hz.

[0150] However, with the conventional method, the visibility of the black bands decreases, but they do not disappear. Also, in order to increase the refresh rate while maintaining the accuracy of the LED display gradation, it is necessary to either increase the frequency of the reference clock used for PWM control of the LED driver, or reduce the number of LED rows that are turned on in one scan.

[0151] Specifically, in PWM control, a control signal is generated by accumulating a reference clock using a counter, so in order to maintain the accuracy of the LED's display gradation and increase the refresh rate, it is necessary, for example, to increase the frequency of the reference clock.

[0152] The frequency f [Hz] of the reference clock is expressed by the following equation (1) using the refresh rate RR [Hz], the display gradation precision t [bits], and the number of LED rows L [rows] that are illuminated in one scan.

[0153] f=RR×2 t ×L (1)

[0154] For example, if the refresh rate is 3840Hz, the display gradation precision is 12 bits, and the number of LED rows that emit light in one scan is 16, the reference clock frequency is approximately 251.7MHz. In contrast, if the refresh rate is twice as fast as 3840Hz, 7680Hz, the reference clock frequency will also be twice as fast as when the refresh rate is 3840Hz, approximately 503.3MHz. As described above, an increase in the reference clock frequency (operating frequency) of the LED driver will lead to increased costs due to changes in the power consumption of the LED driver and the manufacturing process of the LED driver.

[0155] On the other hand, when increasing the refresh rate while maintaining the frequency of the reference clock, it is necessary to reduce the number of LED rows that emit light in one scan, as shown in the above formula (1). In order to reduce the number of LED rows that emit light in one scan, it is necessary to increase the number of LED drivers. For example, if the number of LED rows that emit light in one scan is reduced by half to double the refresh rate, it is necessary to double the number of LED drivers. This leads to increased costs and a shortage of component mounting area on the board. In addition, dark power also increases with the increase in LED drivers. Dark power is the power consumed even when the LEDs are not emitting light, and is mainly consumed by the LED drivers, the control circuits for driving the LED drivers, the power supply circuits, etc.

[0156] As shown in the above formula (1), it is possible to increase the refresh rate while maintaining the frequency of the reference clock and the number of rows of LEDs that are illuminated in one scan by lowering the display grayscale accuracy, or the so-called grayscale. For example, the refresh rate can be doubled by reducing the display grayscale accuracy by one bit. However, a decrease in the display grayscale accuracy reduces the quality of the re-captured image. Although the decrease in the display grayscale accuracy can be suppressed by slightly increasing the frequency of the reference clock, the quality of the re-captured image will decrease in either case.

[0157] In contrast, the re-imaging system 10 can prevent the occurrence of black bars simply by controlling the refresh rate based on the exposure time. Therefore, there is no significant increase in manufacturing costs or deterioration in the accuracy of displayed gradations.

[0158] Note that the refresh rate may be controlled by the user, rather than by the refresh rate control unit 130. In this case, the exposure time is not transmitted from the video camera 11, and the user determines the refresh rate so that the exposure time corresponds to the light emission period of the LED 241, and inputs the determined refresh rate to the PC 62 using the input unit 106. The PC 62 then generates a display control signal representing the refresh rate, etc. input by the user.

[0159] In the re-shooting system 10, the exposure time is transmitted from the video camera 11 to the PC 62, but the user may operate the input unit 106 to input the exposure time of the video camera 11, and the refresh rate control unit 130 may acquire the exposure time.

[0160] In the re-capture system 10, the PC 62 controls the refresh rate based on the exposure time, but the video camera 11 may transmit the exposure time to the video wall controller 63 or the video wall 12, and at least a part of the video wall controller 63 or the video wall 12 may function as the refresh rate control unit 130.

[0161] In the re-imaging system 10, the refresh rate is controlled so that the exposure time of the video camera 11 is an integer multiple of the light emission cycle of the LED 241. However, in practice, there are cases where the vertical blanking period of each frame must also be controlled based on the exposure time. Hereinafter, a re-imaging system that also controls the vertical blanking period will be described as a second embodiment.

[0162] <Second embodiment> <Example of re-imaging system configuration> FIG. 22 is a diagram showing a configuration example of a second embodiment of a re-imaging system including an information processing system to which the present technology is applied.

[0163] In the re-imaging system 400 in Fig. 22, the same reference numerals are used for the parts corresponding to the re-imaging system 10 in Fig. 3. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on the parts that are different from the re-imaging system 10.

[0164] The re-capture system 400 of FIG. 22 differs from the re-capture system 10 in that the PC 62, the video wall controller 63, and the video wall 12 are replaced with a PC 401, a video wall controller 402, and a video wall 403, but is otherwise configured in the same manner as the re-capture system 10.

[0165] Specifically, the PC 401 (information processing system) is a general-purpose computer. The PC 401 generates a control command for controlling the video wall controller 402 and transmits the control command to the video wall controller 402, thereby controlling the video wall controller 402.

[0166] For example, the PC 401 performs a refresh rate control process similar to the PC 62 in Fig. 3. The PC 401 also determines a vertical blanking period of the video displayed on the video wall 403 so that the exposure time is an integer multiple of the light emission cycle (subframe) of the LED 241. Specifically, the PC 401 controls the vertical blanking period by transmitting a vertical blanking period command, which is a control command representing the vertical blanking period, to the video wall controller 402 based on the refresh rate determined by the refresh rate control process.

[0167] 7, but the information represented by the display control signal is different. Specifically, like the video wall controller 63, the video wall controller 402 performs predetermined signal processing on the frame-by-frame video signal supplied from the video server 61, divides it, and transmits it to each display unit 411. The video wall controller 402 also transmits to each display unit 411 display control signals representing the refresh rate represented by the refresh rate command supplied from the PC 401, the vertical blanking period represented by the vertical blanking period command, and the like.

[0168] The video wall 403 (display unit) is a large LED display in which n display units 411-1 to 411-n are arranged in a tiled pattern. Fig. 22 illustrates a case in which n is 6 x 24. In the following, when there is no need to distinguish between the display units 411-1 to 411-n, they will simply be referred to as display units 411.

[0169] The display unit 411 includes m LED arrays 51 arranged in a tiled shape. Each display unit 411 of the video wall 403 is connected to the video wall controller 402. Based on a frame-by-frame video signal and a display control signal supplied from the video wall controller 402, the video wall 403 displays a video corresponding to the video signal in units of frames. Specifically, the video signal supplied to each display unit 411 is a video signal corresponding to the position of that display unit 411 on the video wall 403. Each display unit 411 displays a video based on the video signal in accordance with the display control signal, whereby one frame of video is displayed on the entire video wall 403.

[0170] In the re-imaging system 400 configured as above, similarly to the re-imaging system 10, each device is frame-synchronized by a genlock signal.

[0171] The video wall controller 402 and the video wall 403 may be integrated into one unit, or may be a display device that is integrated with them. The PC 401, the video wall controller 402, and the video wall 403 may be integrated into one unit, or may be a display device that is integrated with them.

[0172] <Configuration example of vertical blanking control section> The hardware configuration of the PC 401 in FIG. 22 is similar to that of the PC 62 in FIG. 5, and therefore a description thereof will be omitted.

[0173] FIG. 23 is a block diagram showing an example of the functional configuration of the CPU when the CPU of the PC 401 in FIG. 22 performs vertical blanking interval control processing for controlling the vertical blanking interval.

[0174] When the CPU of the PC 401 performs vertical blanking interval control processing, it functions as the vertical blanking interval control unit 420 of Fig. 23. The vertical blanking interval control unit 420 is made up of an acquisition unit 421, a control unit 422, and a transmission control unit 423.

[0175] The acquisition unit 421 acquires the refresh rate determined by the refresh rate control process, and supplies the refresh rate to the control unit 422 .

[0176] The control unit 422 determines a vertical blanking period based on the refresh rate supplied from the acquisition unit 421. The control unit 422 controls the vertical blanking period by generating a vertical blanking period command representing the vertical blanking period. The control unit 422 outputs the vertical blanking period command to the transmission control unit 423.

[0177] The transmission control unit 423 supplies the vertical blanking period command supplied from the control unit 422 to a communication unit of the PC 401, and instructs the communication unit to transmit the vertical blanking period command to the video wall controller 402. As a result, the communication unit transmits (outputs) the vertical blanking period command to the video wall controller 402 via a wired connection such as a LAN cable.

[0178] <Example of detailed display unit configuration> FIG. 24 is a block diagram showing an example of a detailed configuration of the display unit 411 in FIG.

[0179] In the display unit 411 in Fig. 24, the parts corresponding to the display unit 41 in Fig. 8 are denoted by the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on the parts different from the display unit 41.

[0180] A display unit 411 in FIG. 24 differs from the display unit 41 in that an LED block 441 is provided instead of the LED block 192, but otherwise has the same configuration as the display unit 41.

[0181] The LED block 441 differs from the LED block 192 in that LED drivers 461-1 to 461-m are provided instead of the LED drivers 221-1 to 221-m, and is otherwise configured similarly to the LED block 192.

[0182] The LED drivers 461-1 to 461-m are connected to the signal output IFs 214-1 to 214-m of the driver control unit 191, respectively, and are also connected to the LED arrays 51-1 to 51-m, respectively. Note that, hereinafter, when it is not necessary to individually distinguish the LED drivers 461-1 to 461-m, they will simply be referred to as the LED driver 461.

[0183] The LED driver 461 drives each LED 241 by PWM controlling the light emission of each LED 241 constituting the LED array 51 connected to it, based on a display control signal and a display signal transmitted from the signal output IF 214 connected to it.

[0184] Specifically, the LED driver 461 generates a PWM signal so that each LED 241 in the LED array 51 emits light at an intensity based on the display signal corresponding to that LED 241 for a refresh rate represented by the display control signal per second. The LED driver 461 supplies the PWM signal to each LED 241. As a result, an image corresponding to the position of that LED array 51 on the video wall 403 is displayed on the LED array 51, among images corresponding to the frame-by-frame video signal output from the video server 61. The LED driver 461 also controls the vertical blanking period based on the display control signal so that a vertical blanking period represented by the display control signal is provided for each frame.

[0185] <Description of vertical blanking period> FIG. 25 is a diagram illustrating the vertical blanking period of the video displayed on the video wall 403. As shown in FIG.

[0186] In Fig. 25, the horizontal axis represents the light emission time. The top row represents the vertical synchronization signal (V-Sync) of the video displayed on the video wall 403. The second row from the top onwards represents the light emission timing of each of the 30 rows of LEDs 241 from the top of the video wall 403, and in Fig. 25, the i-th row from the top is represented as Line#i. In the example of Fig. 25, the number of rows of LEDs 241 constituting the LED array 51 is 15, and the number of scans within one frame period is 32. The same applies to Fig. 27 described later.

[0187] As shown in FIG. 25, between vertical synchronization signals, that is, during one frame period, a vertical blanking period (V-blanking) in which light emission of all the LEDs 241 is stopped after all scans are completed is provided.

[0188] Specifically, first, in synchronization with a vertical synchronization signal, each LED driver 461 generates a PWM signal corresponding to the display signal (e.g., image data with values ​​from 0 to 2048) of each LED 241 for each row, starting from the LED 241 in the top row of the LED array 51 connected to itself, and supplies the PWM signal to each LED 241. As a result, light emission begins in order from the LED 241 in the top row in each LED array 51. In the example of FIG. 25, light emission begins simultaneously from the LED 241 in the first row and the 16th row from the top of the video wall 12. When light emission of the LED 241 in the bottom row of each LED array 51 (in the example of FIG. 25, the 15th row and the 30th row from the top of the video wall 12) ends, that is, when one scan ends, scanning is performed again. Thereafter, scanning is performed in the same manner, 32 times in the example of FIG. 25. That is, the LED driver 461 generates a PWM signal of the same PWM width 32 times for each LED 241. Thereafter, a vertical blanking period starts until the vertical synchronization signal for the next frame is generated, during which all the LEDs 241 stop emitting light.

[0189] <Explanation of the cause of black belt occurrence> FIG. 26 is a diagram for explaining the cause of black bars appearing in an image re-captured by the re-capture system when the exposure time is not an integer multiple of the LED light emission period.

[0190] In Fig. 26, the horizontal axis represents the light emission time. The top row represents the vertical synchronization signal of the video displayed on the video wall. The second row from the top onwards represents the light emission timing of each of the top 30 rows of LEDs on the video wall, and in Fig. 26 the i-th row from the top is represented as Line#i. In the example of Fig. 26, the LED array consists of 15 rows of LEDs, and one frame period is scanned 32 times.

[0191] The scanning is the same as in the case of Figure 25, so the explanation will be omitted. The video camera takes pictures using the rolling shutter method with a specified exposure time. Therefore, exposure to the light from the LEDs of each row is performed in order from the top row, and the exposure start timing of each row is different. Note that each device in the re-imaging system is frame synchronized, so the exposure start time and the display start time of one frame are synchronized.

[0192] In such a case, when the exposure time of the video camera does not correspond to the light emission cycle of the LED, the light emission period within the exposure period may not be uniform for each row. In the example of Fig. 26, in the first and second rows from the top, the entire period of the first light emission is included in the exposure period 481. However, in the 15th row from the top, the exposure period 481 does not include a period 482 that is 1 / 2 (0.5) times the entire period of the first light emission. In the 16th row from the top, the exposure period 481 does not include a period 483 that is 0.8 times the entire period of the first light emission.

[0193] Therefore, the luminance of the pixels in the 15th and 16th rows from the top, which are re-captured by the video camera, is lower than that of the pixels in the 1st and 2nd rows from the top. For example, when the luminance of the pixels in the 1st and 2nd rows from the top is set to 1, the luminance of the pixels in the 15th row from the top becomes 0.5, and the luminance of the pixels in the 16th row from the top becomes 0.2. As a result, in the video re-captured by the video camera, the videos of the 15th and 16th rows from the top become black bands. Since a similar situation occurs periodically in the subsequent rows, light and dark appear periodically in the video re-captured by the video camera.

[0194] <Explanation of PC Control> FIG. 27 is a diagram for explaining the control by the PC 401.

[0195] As shown in FIG. 27, the control unit 132 of the PC 401 determines the refresh rate so that the exposure time and the emission cycle of the video camera 11 correspond. Specifically, for example, the control unit 132 synchronizes the exposure start time and the scan start time, and controls the refresh rate so that the exposure time is an integer multiple of the emission cycle.

[0196] In the example of FIG. 27, the control unit 132 determines the refresh rate so that the exposure time is 1 times the emission cycle. As a result, compared with the case of FIG. 26, the scan speed becomes faster, and the number of scans within the exposure period becomes 1 for all rows. For example, in the 15th row from the top, the entire period of the first emission is included within the exposure period 481. In the 16th row from the top, a partial period 491 of the entire period of the first emission and a partial period 492 of the entire period of the second emission are included within the exposure period 481. The sum of the period 491 and the period 492 is the entire period of one emission. Therefore, the luminance of the pixels in all rows re-captured by the video camera 11 becomes the same. As a result, in the video in frame units re-captured by the video camera 11, rows of pixels with low luminance do not occur, and black bands do not occur.

[0197] In addition, in the 16th row from the top, when the PWM width is small, sufficient brightness can be obtained with only the exposure by the second emission.

[0198] As described above, when preventing the occurrence of black bars by increasing the scan speed compared to the case of Fig. 26, the end time of the final 32nd scan will be earlier than in the case of Fig. 26. Therefore, compared to the case of Fig. 26, the period 493 from the end of the final scan to the start of display of the next frame will be longer by period 494, and the vertical blanking period must be longer by period 494. In other words, the vertical blanking period must be controlled based on the refresh rate.

[0199] Therefore, the control unit 422 in Fig. 23 calculates the period from the time when the number of scans in one frame section corresponding to the refresh rate is performed until the display of the next frame starts, based on the refresh rate, and determines it as the vertical blanking period. If the vertical blanking period is long, the luminance may become dark.

[0200] The horizontal distance (time) of the inclination of the exposure period 481 represents the curtain speed, which is the time from when the exposure of the first row starts to when the exposure of the last row starts. This curtain speed differs depending on the characteristics and angle of view of the shooting section of the video camera 11.

[0201] In the example of Figure 27, the refresh rate is controlled to be higher than in the example of Figure 26, but the refresh rate may be controlled to be lower than in the example of Figure 26 as long as it is controlled so that the exposure time and the light emission period correspond to each other.

[0202] <Description of Vertical Blanking Period Control Process> Fig. 28 is a flowchart explaining the vertical blanking interval control process of the vertical blanking interval control unit 420 of Fig. 23. This vertical blanking interval control process is started, for example, when the refresh rate is determined by the control unit 132 of the refresh rate control unit 130.

[0203] In step S201 of FIG. 28, the acquisition unit 421 acquires the refresh rate from the control unit 132 and supplies the refresh rate to the control unit 422.

[0204] In step S202, the control unit 422 calculates the period from the time when the number of scans in one frame section corresponding to the refresh rate is performed until the display of the next frame starts based on the refresh rate supplied from the acquisition unit 421 in step S201, and determines it as a vertical blanking period. The control unit 422 generates a vertical blanking period command representing the vertical blanking period, and supplies it to the transmission control unit 423.

[0205] In step S203, the transmission control unit 423 supplies the vertical blanking period command supplied from the control unit 422 in step S202 to the communication unit 109 and instructs the communication unit 109 to transmit the vertical blanking period command to the video wall controller 402. Then, the process ends.

[0206] As described above, the re-capture system 400 controls the vertical blanking period of the image displayed on the video wall 403 based on the exposure time. Specifically, the re-capture system 400 determines the refresh rate based on the exposure time so that the exposure time is an integer multiple of the light emission cycle, and controls the vertical blanking period based on the refresh rate. Therefore, it is possible to prevent the occurrence of black bars when re-capturing the video wall 403.

[0207] In the above-mentioned first and second embodiments, the exposure period is within the display period of one frame of video, but depending on the exposure time, the exposure period may span the display period of two frames of video. That is, the exposure period may include a vertical blanking period. In this case, black bars may occur because the number of vertical blanking periods included in the exposure period differs depending on the row. Below, a re-imaging system that prevents the occurrence of such black bars will be described as a third embodiment.

[0208] <Third embodiment> <Example of re-imaging system configuration> FIG. 29 is a diagram showing an example configuration of a re-imaging system including an information processing system to which the present technology is applied according to a third embodiment.

[0209] In the re-imaging system 500 in Fig. 29, the same reference numerals are used for the parts corresponding to the re-imaging system 10 in Fig. 3. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on the parts that are different from the re-imaging system 10.

[0210] The re-capture system 500 of Figure 29 differs from the re-capture system 10 in that the PC 62, video wall controller 63, and video wall 12 are replaced by a PC 501, a video wall controller 502, and a video wall 503, but is otherwise configured in the same manner as the re-capture system 10.

[0211] Specifically, the PC 501 (information processing system) is a general-purpose computer. The PC 501 generates a control command for controlling the video wall controller 502 and transmits the control command to the video wall controller 502, thereby controlling the video wall controller 502.

[0212] For example, PC 501 sets a control mode for controlling the vertical blanking period of the image displayed on video wall 503 based on the exposure time and curtain speed of video camera 11, and controls the vertical blanking period by transmitting a control mode command representing the control mode to video wall controller 502. The curtain speed of video camera 11 is input by the user, for example.

[0213] 7, but the information represented by the display control signal is different. Specifically, like the video wall controller 63, the video wall controller 502 performs predetermined signal processing on the frame-by-frame video signal supplied from the video server 61, divides it, and transmits it to each display unit 511. The video wall controller 502 also transmits to each display unit 511 a display control signal indicating a control mode, etc., represented by a control mode command supplied from the PC 501.

[0214] The video wall 503 (display unit) is a large LED display in which n display units 511-1 to 511-n are arranged in a tiled pattern. Fig. 29 illustrates a case in which n is 6 x 24. In the following, when there is no need to distinguish between the display units 511-1 to 511-n, they will simply be referred to as display units 511.

[0215] The display unit 511 includes m LED arrays 51 arranged in a tiled shape. Each display unit 511 of the video wall 503 is connected to the video wall controller 502. Based on a frame-by-frame video signal and a display control signal supplied from the video wall controller 502, the video wall 503 displays a video corresponding to the video signal in units of frames. Specifically, the video signal supplied to each display unit 511 is a video signal corresponding to the position of the display unit 511 on the video wall 503. Each display unit 511 displays a video based on the video signal in accordance with the display control signal, whereby one frame of video is displayed on the entire video wall 503.

[0216] In the re-imaging system 500 configured as above, similarly to the re-imaging system 10, each device is frame-synchronized by a genlock signal.

[0217] The video wall controller 502 and the video wall 503 may be integrated into one unit, or may be a display device that is integrated with them. The PC 501, the video wall controller 502, and the video wall 503 may be integrated into one unit, or may be a display device that is integrated with them.

[0218] <Configuration example of vertical blanking control section> The hardware configuration of the PC 501 in FIG. 29 is similar to that of the PC 62 in FIG. 5, and therefore a description thereof will be omitted.

[0219] FIG. 30 is a block diagram showing an example of the functional configuration of the CPU when the CPU of the PC 501 in FIG. 29 performs vertical blanking interval control processing for controlling the vertical blanking interval.

[0220] When the CPU of the PC 501 performs vertical blanking interval control processing, it functions as the vertical blanking interval control unit 520 in Fig. 30. The vertical blanking interval control unit 520 is made up of an acquisition unit 521, a control unit 522, and a transmission control unit 523.

[0221] The acquisition unit 521 acquires the exposure time transmitted from the setting unit 82 of the video camera 11 and received via the communication unit of the PC 501. The acquisition unit 521 supplies the exposure time to the control unit 522.

[0222] The control unit 522 sets a control mode for the vertical blanking period based on the exposure time and the curtain speed of the video camera 11 supplied from the acquisition unit 521. The control unit 522 controls the vertical blanking period by generating a control mode command representing the control mode. The control unit 522 outputs the control mode command to the transmission control unit 523.

[0223] The transmission control unit 523 supplies the control mode command supplied from the control unit 522 to the communication unit of the PC 501, and instructs the communication unit to transmit the control mode command to the video wall controller 502. As a result, the communication unit transmits (outputs) the control mode command to the video wall controller 502 via a wired connection such as a LAN cable.

[0224] <Example of detailed display unit configuration> FIG. 31 is a block diagram showing an example of a detailed configuration of the display unit 511 in FIG.

[0225] In the display unit 511 in Fig. 31, the parts corresponding to the display unit 41 in Fig. 8 are denoted by the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on the parts that are different from the display unit 41.

[0226] A display unit 511 in FIG. 31 differs from the display unit 41 in that an LED block 541 is provided instead of the LED block 192, but otherwise has the same configuration as the display unit 41.

[0227] The LED block 541 differs from the LED block 192 in that LED drivers 561-1 to 561-m are provided instead of the LED drivers 221-1 to 221-m, and is otherwise configured similarly to the LED block 192.

[0228] The LED drivers 561-1 to 561-m are connected to the signal output IFs 214-1 to 214-m of the driver control unit 191, respectively, and are also connected to the LED arrays 51-1 to 51-m, respectively. Note that, hereinafter, when it is not necessary to individually distinguish the LED drivers 561-1 to 561-m, they will simply be referred to as the LED driver 561.

[0229] The LED driver 561 drives each LED 241 by PWM controlling the light emission of each LED 241 constituting the LED array 51 connected to it, based on a display control signal and a display signal transmitted from the signal output IF 214 connected to it.

[0230] Specifically, the LED driver 561 controls the timing of starting emission of each frame based on the control mode indicated by the display control signal. The LED driver 561 generates a PWM signal so that each LED 241 of the LED array 51 emits light at an intensity based on the display signal corresponding to that LED 241 for a predetermined refresh rate per second, and supplies the PWM signal to each LED 241. As a result, the LED array 51 displays an image corresponding to the position of that LED array 51 on the video wall 503, among images corresponding to the frame-by-frame video signal output from the video server 61.

[0231] <Description of vertical blanking period> FIG. 32 is a diagram illustrating the vertical blanking period of the video displayed on the video wall 503. As shown in FIG.

[0232] The upper diagram of Fig. 32 shows the light emission timing of the LEDs 241 in each of the 1st to 135th rows from the top of the video wall 503. In the upper diagram of Fig. 32, the horizontal axis shows the light emission time, and the vertical axis shows each row of the LEDs 241. In the example of Fig. 32, the frame rate of the video wall 503 is 120 Hz, and one frame period is 1 / 119.88 seconds. The number of rows of the LEDs 241 constituting the LED array 51 is 15. The lower diagram of Fig. 32 shows the light emission timing of the LEDs 241 in the 135th row with a line.

[0233] As shown in the upper diagram of Figure 32, a vertical blanking period (V-blanking) is provided after the completion of all scans in one frame section. In the upper diagram of Figure 32, the interval between the diagonal lines connecting the light emission timing of each row is the time for one scan.

[0234] <Exposure time and vertical blanking period explanation> FIG. 33 is a diagram illustrating the exposure time and the vertical blanking period.

[0235] The diagram on the left side of Fig. 33 shows the light receiving surface of the shooting section 85 of the video camera 11. The diagram on the right side of Fig. 33 shows images displayed on the video wall 503 corresponding to the pixels of each row of the light receiving surface of the shooting section 85. The horizontal axis of the diagram on the right side of Fig. 33 shows the display time of the image, and the vertical axis shows the row number from the top of the row of pixels of the light receiving surface of the shooting section 85 corresponding to the image displayed at that display time. In the example of Fig. 33, the frame rate of the video wall 503 is 120 Hz, and the frame rate of the video camera 11 is 60 Hz.

[0236] In the example of Fig. 33, as shown in the right diagram of Fig. 33, when the exposure time, i.e., the opening angle, is 360 degrees, exposure is performed with light corresponding to two frames of video displayed on the video wall 503. When the opening angle is 270 degrees, exposure is performed with light corresponding to 1.5 frames of video, and when the opening angle is 180 degrees, exposure is performed with light corresponding to one frame of video. When the opening angle is 90 degrees, exposure is performed with light corresponding to 0.5 frames of video.

[0237] Here, since the opening angle is in the range of 0 to 360 degrees, the image corresponding to the light used for exposure of the video camera 11 can be 0 to 2 frames of image. In other words, the exposure period can include 0 to 2 vertical blanking periods.

[0238] <Explanation of the cause of black belt occurrence> FIG. 34 is a diagram for explaining the cause of black bars appearing in an image re-captured by the re-capture system when the vertical blanking interval is not controlled.

[0239] The horizontal axis of Fig. 34 represents the display time of the image, and the vertical axis represents the row number from the top of the row of pixels on the light receiving surface of the shooting unit corresponding to the image displayed at that display time. In the example of Fig. 34, the frame rate of the video wall is 120 Hz, and the frame rate of the video camera is 60 Hz.

[0240] 34, when the opening angle is 270 degrees, rows above the Lth row on the light receiving surface of the imaging unit include one vertical blanking period within exposure period 571, but rows below the Lth row include two vertical blanking periods within exposure period 571. When the opening angle is 90 degrees, rows above the Lth row on the light receiving surface of the imaging unit 85 do not include a vertical blanking period within exposure period 571, but rows below the Lth row include one vertical blanking period within exposure period 571.

[0241] Here, no light is emitted during the vertical blanking period. Therefore, when the opening angle is 270 degrees or 90 degrees, the luminance of the pixels in the rows below the Lth row re-captured by the video camera is lower than the luminance of the pixels in the rows above the Lth row. As a result, the images re-captured by the video camera have a black belt in the rows below the Lth row.

[0242] When the opening angle is 270 degrees or 90 degrees, even if the light emission start timing is shifted, the number of vertical blanking periods included in the exposure period 571 or 572 always differs from row to row. As a result, black bands appear in the image re-captured by the video camera.

[0243] On the other hand, when the opening angle is 180 degrees, for all rows of the light receiving surface of the imaging section, one vertical blanking period is included within the exposure period 571. Although not shown, when the opening angle is 360 degrees, for all rows of the light receiving surface of the imaging section 85, two vertical blanking periods are included within the exposure period 571. Therefore, when the opening angle is 180 degrees or 360 degrees, no black bands are generated.

[0244] <Control mode explanation> 35 and 36 are diagrams for explaining the control modes.

[0245] The horizontal axis in Figures 35 and 36 represents the display time of the image, and the vertical axis represents the row number from the top of the row of pixels on the light receiving surface of the imaging unit 85 that is exposed to light corresponding to the image displayed at that display time. In the example of Figures 35 and 36, the frame rate of the video wall 503 is 120 Hz, and the frame rate of the video camera 11 is 60 Hz.

[0246] As shown in FIG. 35, for example, when the opening angle is 90 degrees and the exposure period of some rows of the imaging section 85 does not span a vertical blanking period, and the exposure period of the remaining rows spans one vertical blanking period, the control section 522 sets the control mode to a first mode in which the vertical blanking period of the even frames is not included in the exposure period of any row of the imaging section 85.

[0247] When the control mode is set to the first mode, the LED driver 561 advances the light emission start timing (video display start time) of an even frame (second frame) that is the next frame of an odd frame (first frame) among adjacent frames, thereby deleting the vertical blanking period of the odd frame. As a result, the display period 581 of the odd frame is shortened by the vertical blanking period, and the display period 582 of the even frame is extended by the vertical blanking period.

[0248] As described above, when the vertical blanking period of the odd-numbered frames is deleted, the vertical blanking period is not included in the period T from when the exposure of the first row from the top of the shooting section 85 starts to when the exposure of the last row ends (when the shutter is released), that is, the period of the sum of the curtain speed and the exposure time. Therefore, the vertical blanking period is not included in the exposure period of any row of the shooting section 85. As a result, in the video signal re-captured by the video camera 11 in frame units, no rows of pixels with low brightness are generated, and no black bands are generated.

[0249] On the other hand, as shown in FIG. 36, for example, when the opening angle is 270 degrees and the exposure period of some rows of the imaging section 85 spans one vertical blanking period and the exposure period of the remaining rows spans two vertical blanking periods, the control unit 522 sets the control mode to a second mode in which the vertical blanking periods of even frames are included in the exposure periods of all rows of the imaging section 85.

[0250] When the control mode is set to the second mode, the LED driver 561 deletes the vertical blanking period of the odd frames, as in the first mode. This shortens the display period 591 of the odd frames by the vertical blanking period, and extends the display period 592 of the even frames by the vertical blanking period. Then, the light emission start times (display start times) of all frames are changed to be later by the time D so that the vertical blanking period of the even frames is included in the exposure period 571 of all rows of the imaging section 85.

[0251] As a result, the vertical blanking period of the even frames is included in the exposure period 571 of all rows of the imaging section 85. As a result, in the frame-by-frame video signal recaptured by the video camera 11, no rows of pixels with low brightness are generated, and no black bands are generated.

[0252] The light emission start times of all frames can be shifted by up to one frame section, for example. For example, when the frame rate is 120 Hz, the light emission start times of all frames can be shifted by up to 1 / 120 seconds.

[0253] The control unit 522 determines whether the exposure period of only some of the rows of the imaging unit 85 spans one vertical blanking period or two vertical blanking periods based on the curtain speed and the exposure time. If the control unit 522 determines that the exposure period of only some of the rows spans one vertical blanking period, it sets the control mode to the first mode. If the control unit 522 determines that the exposure period of only some of the rows spans two vertical blanking periods, it sets the control mode to the second mode.

[0254] <Description of Vertical Blanking Period Control Process> Fig. 37 is a flowchart for explaining the vertical blanking interval control process of the vertical blanking interval control unit 520 of Fig. 30. This vertical blanking interval control process is started, for example, when an exposure time is transmitted from the setting unit 82 of the video camera 11.

[0255] In step S331 of FIG. 37, the acquisition unit 521 acquires the exposure time transmitted from the setting unit 82 and received via the communication unit of the PC 501, and supplies the exposure time to the control unit 522.

[0256] In step S332, control unit 522 determines whether or not it is necessary to delete a vertical blanking period based on the exposure time and the curtain speed of video camera 11 supplied from acquisition unit 521 in the process of step S331. Specifically, control unit 522 determines whether or not the vertical blanking periods included in the exposure periods of all rows of imaging unit 85 are the same based on the exposure time and the curtain speed. When control unit 522 determines that the vertical blanking periods included in the exposure periods of all rows are not the same, it determines that it is necessary to delete a vertical blanking period and proceeds to the process of step S33.

[0257] In step S333, the control unit 522 sets the control mode based on the exposure time and the curtain speed. Specifically, the control unit 522 determines whether the exposure period of some rows of the shooting unit 85 does not span a vertical blanking period and the exposure period of the remaining rows spans one vertical blanking period based on the exposure time and the curtain speed. If the control unit 522 determines that the exposure period of some rows does not span a vertical blanking period and the exposure period of the remaining rows spans one vertical blanking period, the control unit 522 sets the control mode to the first mode. On the other hand, if it is determined that the exposure period of some rows does not span a vertical blanking period and the exposure period of the remaining rows does not span one vertical blanking period, that is, if the exposure period of some rows spans one vertical blanking period and the exposure period of the remaining rows spans two vertical blanking periods, the control unit 522 sets the control mode to the second mode.

[0258] The control unit 522 controls the vertical blanking period by generating a control mode command representing the set control mode. The control unit 522 outputs the control mode command to the transmission control unit 523.

[0259] In step S334, the transmission control unit 523 supplies the control mode command supplied from the control unit 522 in step S333 to the communication unit of the PC 501 and instructs the communication unit to transmit the control mode command to the video wall controller 502. Then, the process ends.

[0260] On the other hand, if the control unit 522 determines in step S332 that the vertical blanking periods included in the exposure periods of all the rows are the same, the process ends.

[0261] As described above, the recapture system 500 controls the vertical blanking period of the image displayed on the video wall 503 based on the exposure time. Therefore, it is possible to prevent the occurrence of black bars when the video wall 503 is recaptured.

[0262] Although the re-imaging system 500 does not perform the refresh rate control process, the refresh rate control process may be performed. In this case, the refresh rate control process differs depending on whether or not a vertical blanking period is included in the exposure period. The refresh rate control process will be described below.

[0263] <Explanation of refresh rate control process> 38 and 39 are diagrams for explaining the refresh rate control process when the re-imaging system 500 performs the refresh rate control process.

[0264] 38 and 39, the horizontal axis represents the light emission time. The top row represents the vertical synchronization signal of the video displayed on video wall 503. The second row from the top onwards represents the light emission timing of each of the 30 rows of LEDs 241 from the top of video wall 503, and in Figs. 38 and 39, the i-th row from the top is represented as Line#i. In Figs. 38 and 39, the number of rows of LEDs 241 constituting LED array 51 is 15.

[0265] When the exposure period 601 does not include a vertical blanking period after the vertical blanking control process, as shown in Fig. 38, the PC 501 (control unit) determines the refresh rate so that the exposure time is an integer multiple of the light emission period of the LED 241. In the example of Fig. 38, the exposure time is 7 times the light emission period.

[0266] On the other hand, when one vertical blanking period is included in the exposure period 602 after the vertical blanking control process, the PC 501 determines the refresh rate so that the exposure time is the sum of an integer multiple (N times) of the light emission cycle of the LED 241 and one vertical blanking period, as shown in Fig. 39. In the example of Fig. 38, the exposure time is the sum of 34 times the light emission cycle and one vertical blanking period, and the number of scans within one frame section is 32.

[0267] Although not shown, when the exposure period includes two vertical blanking periods, the refresh rate is determined so that the exposure time is the sum of an integer multiple of the light emission cycle of the LED 241 and two vertical blanking periods.

[0268] When the PC 501 performs the above-described refresh rate control process, the vertical blanking period may be further controlled based on the refresh rate, as in the second embodiment.

[0269] <Description of vertical blanking period control based on refresh rate> FIG. 40 is a diagram for explaining the control of the vertical blanking period based on the refresh rate by the PC 501 in this case.

[0270] In Fig. 40, the horizontal axis represents the light emission time. The top row represents the vertical synchronization signal of the video displayed on video wall 503. The second row from the top onwards represents the light emission timing of each of the 30 rows of LEDs 241 from the top of video wall 503, and in Fig. 40, the i-th row from the top is represented as Line#i. In the example of Fig. 40, the number of rows of LEDs 241 constituting LED array 51 is 15.

[0271] As shown in FIG. 40, when one vertical blanking period is included in an exposure period 611, the PC 501 controls the vertical blanking period so that the exposure period becomes the sum of an integer multiple of the light emission cycle of the LED 241 and one vertical blanking period.

[0272] Specifically, first, the PC 501 determines the refresh rate so that the exposure period is the sum of an integer multiple of the light emission cycle of the LED 241 and one vertical blanking period.

[0273] In the example of FIG. 40, the PC 501 determines the refresh rate so that the exposure time is the sum of 33 times the light emission period and one vertical blanking period. This results in 33 scans in the exposure period for all rows. For example, in the 17th row from the top, a period 621 that is a part of the entire period of the first light emission and a period 622 that is a part of the entire period of the last light emission are included in the exposure period 611. The sum of the periods 621 and 622 is the entire period of one light emission. Therefore, the luminance of the pixels in all rows recaptured by the video camera 11 is the same. As a result, in the frame-by-frame image recaptured by the video camera 11, no rows of pixels with low luminance are generated, and no black bands are generated.

[0274] Next, based on the determined refresh rate, PC 501 calculates a period 623 from the end of the last scan in one frame section to the start of display (light emission) of the next frame, and controls vertical blanking by determining that period 623 as the vertical blanking period.

[0275] On the other hand, although not shown in the figure, when a vertical blanking period is not included in the exposure period, similarly to the second embodiment, the vertical blanking period is controlled so that the exposure period becomes an integer multiple of the light emission cycle of the LED 241. Specifically, the PC 401 controls vertical blanking based on a refresh rate determined so that the exposure period becomes an integer multiple of the light emission cycle of the LED 241.

[0276] In the re-imaging system 500, the maximum number of vertical blanking periods included in an exposure period is 2, but this number is not limited to 2. Even if the maximum number of vertical blanking periods included in an exposure period is 2 or more, the occurrence of black bars can be prevented by performing the same process as in the case of 2.

[0277] In the re-shooting system 400 (500), the exposure time is transmitted from the video camera 11 to the PC 401 (501), but the user may operate the input section of the PC 401 (501) to input the exposure time of the video camera 11, and the refresh rate control section 130 (vertical blanking period control section 520) may acquire the exposure time.

[0278] In the re-shooting system 400 (500), the PC 401 (501) controls the refresh rate and the vertical blanking period, but the video camera 11 may transmit the exposure time to the video wall controller 402 (502) or the video wall 403 (503), and at least a part of the video wall controller 402 (502) or the video wall 403 (503) may function as the refresh rate control unit 130 or the vertical blanking period control unit 420 (520).

[0279] In the first to third embodiments, the PC 62 (401, 501) exchanges signals with the video camera 11 via a wire such as a coaxial cable, but may exchange signals via wireless communication. Similarly, the PC 62 (401, 501) exchanges signals with the video wall controller 63 (402, 502) via a wire such as a LAN cable, but may exchange signals via wireless communication.

[0280] The above-mentioned series of processes can be executed by hardware or software.

[0281] The re-imaging system 10 (400, 500) may be provided with multiple video wall controllers 63 (402, 502), and each video wall controller 63 (402, 502) may divide and control the video wall 12 (403, 503). If the video wall 12 (403, 503) is composed of one display unit 41 (411, 511), the video wall controller 63 (402, 502) may not be provided. In this case, the frame-by-frame video signal output from the video server 61 and the control command output from the PC 62 (401, 501) are directly input to the video wall 12 (403, 503).

[0282] In this specification, a system refers to a collection of multiple components (devices, modules (parts), processes, etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, a single device in which multiple modules are housed in a single case, and a processing unit that performs multiple processes are all systems.

[0283] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0284] For example, it is possible to adopt a configuration in which all or part of the above-described multiple embodiments are combined.

[0285] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0286] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0287] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0288] It should be noted that the effects described in this specification are merely examples and are not limiting, and there may be effects other than those described in this specification.

[0289] The present technology can take the following configurations. (1) A control unit that controls a vertical blanking period of an image displayed on the display unit based on an exposure time of an image capturing unit that captures the display unit. An information processing system comprising: (2) The control unit deletes the vertical blanking period of a first frame, which is one of two adjacent frames of the image, based on the exposure time. It was configured as The information processing system according to (1) above. (3) The control unit advances a display start time of a second frame that is a frame subsequent to the first frame, thereby deleting the vertical blanking period of the first frame. It was configured as The information processing system according to (2) above. (4) The control unit extends the vertical blanking period of the second frame by the vertical blanking period of the first frame. It was configured as The information processing system according to (3) above. (5) The control unit changes display start times of all frames of the image based on the exposure time, thereby including the vertical blanking period of the second frame within an exposure period of all rows of the image. It was configured as The information processing system according to (3) or (4). (6) The control unit includes the vertical blanking period of the second frame within the exposure period of all rows of the image based on a curtain speed of the imaging unit and the exposure time. It was configured as The information processing system according to (5) above. (7) The control unit sets the current mode to a first mode in which the vertical blanking period of the second frame is not included in the exposure period of all rows of the image, or to a second mode in which the vertical blanking period of the second frame is included in the exposure period of all rows of the image, based on the curtain speed and the exposure time of the imaging unit. It was configured as The information processing system according to (6) above. (8) When the control unit sets the current mode to the second mode, the control unit includes the vertical blanking period of the second frame in the exposure period of all rows of the image. It was configured as The information processing system according to (7) above. (9) The control unit controls the vertical blanking period so that the exposure time is an integer multiple of a light emission period of the display unit. It was configured as The information processing system according to (1) above. (10) When the vertical blanking period is not included in an exposure period of the image capturing unit, the control unit controls the vertical blanking period so that the exposure time becomes an integer multiple of a light emission period of the display unit. It was configured as The information processing system according to (9) above. (11) When the vertical blanking period is not included in the exposure period, the control unit determines a refresh rate of the display unit so that the exposure time is an integer multiple of a light emission cycle of the display unit, and controls the vertical blanking period based on the refresh rate. It was configured as The information processing system according to (10) above. (12) When the vertical blanking period is included in an exposure period of the image capturing unit, the control unit controls the vertical blanking period so that the exposure time becomes a sum of an integer multiple of a light emission cycle of the display unit and the vertical blanking period. It was configured as The information processing system according to (9) or (10) above. (13) When the vertical blanking period is included in the exposure period, the control unit determines a refresh rate such that the exposure time is a sum of an integer multiple of a light emission cycle of the display unit and the vertical blanking period, and controls the vertical blanking period based on the refresh rate. It was configured as The information processing system according to (12) above. (14) An acquisition unit for acquiring the exposure time Further equipped The information processing system according to any one of (1) to (13). (15) an input unit for acquiring a video signal to be displayed on the display unit, which is composed of a plurality of display units; a division unit that divides the video signal into video signals to be displayed on each display unit of the display unit; an output section for outputting a control signal for controlling the vertical blanking period and the divided video signal to the display unit; Further equipped with The control unit controls the vertical blanking period by generating the control signal. It was configured as The information processing system according to any one of (1) to (14). (16) The control unit is provided on a first device, the input unit, the division unit, and the output unit are provided on a second device different from the first device; The control unit outputs the control signal to the second device. It was configured as The information processing system according to (15) above. (17) The control unit, the input unit, the division unit, and the output unit are provided on the same device. It was configured as The information processing system according to (15) above. (18) the display unit configured to be driven in accordance with control of the vertical blanking period by the control unit; Further equipped The information processing system according to any one of (1) to (17). (19) the display unit emits light by a passive matrix driving method, The photographing unit photographs using a rolling shutter method. It was configured as The information processing system according to any one of (1) to (18). (20) The information processing system A control step of controlling a vertical blanking period of an image displayed on the display unit based on an exposure time of an image capturing unit that captures the display unit. An information processing method comprising: [Explanation of symbols]

[0290] 11 video camera, 131 acquisition unit, 152 HDMI terminal, 153 DP terminal, 154 DVI terminal, 160 distribution unit, 161-1 to 161-n signal output IF, 400 re-imaging system, 401 PC, 402 video wall controller, 403 video wall, 411 display unit, 422 control unit, 500 re-imaging system, 501 PC, 502 video wall controller, 503 video wall, 511 display unit, 521 acquisition unit, 522 control unit

Claims

1. An acquisition unit that acquires an exposure time of an image capturing unit that captures an image of a display unit by a rolling shutter method; a control unit that controls a vertical blanking period of an image displayed on the display unit that emits light by a passive matrix driving method based on the exposure time; The display unit Equipped with The control unit controls the vertical blanking period so that the exposure time is an integer multiple of a light emission period of the display unit. It was configured as Information processing system.

2. The control unit performs control to delete the vertical blanking period of a first frame, which is one of two adjacent frames of the image, based on the exposure time. It was configured as The information processing system according to claim 1 .

3. The control unit performs control so as to delete the vertical blanking period of the first frame by advancing a display start time of a second frame which is a frame subsequent to the first frame. It was configured as The information processing system according to claim 2 .

4. The control unit controls the vertical blanking period of the second frame to be extended by the vertical blanking period of the first frame. It was configured as The information processing system according to claim 3 .

5. The control unit controls so that the vertical blanking period of the second frame is included in an exposure period of all rows of the imaging unit by changing a display start time of all frames of the image based on the exposure time. It was configured as The information processing system according to claim 3 .

6. The acquisition unit acquires a curtain speed of the shooting unit. It was configured as 6. The information processing system according to claim 5.

7. The control unit performs control based on the curtain speed and the exposure time of the imaging unit so that the vertical blanking period of the second frame is included in the exposure period of all rows of the imaging unit. It was configured as 7. The information processing system according to claim 6.

8. The control unit sets the current mode to a first mode in which the vertical blanking period of the second frame is not included in the exposure period of all rows of the imaging unit, or to a second mode in which the vertical blanking period of the second frame is included in the exposure period of all rows of the imaging unit, based on the curtain speed and the exposure time of the imaging unit. It was configured as The information processing system according to claim 7.

9. When the control unit sets the current mode to the second mode, the control unit controls the vertical blanking period of the second frame to be included in the exposure period of all rows of the imaging unit. It was configured as The information processing system according to claim 8.

10. When the vertical blanking period is not included in an exposure period of the image capturing unit, the control unit controls the vertical blanking period so that the exposure time becomes an integer multiple of a light emission period of the display unit. It was configured as The information processing system according to claim 1 .

11. When the vertical blanking period is not included in the exposure period, the control unit determines a refresh rate of the display unit so that the exposure time is an integer multiple of a light emission cycle of the display unit, and controls the vertical blanking period based on the refresh rate. It was configured as The information processing system according to claim 10.

12. When the vertical blanking period is included in an exposure period of the image capturing unit, the control unit controls the vertical blanking period so that the exposure time becomes a sum of an integer multiple of a light emission cycle of the display unit and the vertical blanking period. It was configured as The information processing system according to claim 1 .

13. When the vertical blanking period is included in the exposure period, the control unit determines a refresh rate of the display unit so that the exposure time is a sum of an integer multiple of a light emission cycle of the display unit and the vertical blanking period, and controls the vertical blanking period based on the refresh rate. It was configured as The information processing system according to claim 12.

14. an input unit for acquiring a video signal to be displayed on the display unit, which is configured with a plurality of display units; a division unit that divides the video signal into video signals to be displayed on each of the display units of the display unit; an output section for outputting a control signal for controlling the vertical blanking period and the divided video signal to the display unit; Further equipped with The control unit controls the vertical blanking period by generating the control signal. It was configured as 14. An information processing system according to any one of claims 1 to 13.

15. The acquisition unit and the control unit are provided on a first device, the input unit, the division unit, and the output unit are provided on a second device different from the first device; The control unit outputs the control signal to the second device. It was configured as 15. The information processing system according to claim 14.

16. The acquisition unit, the control unit, the input unit, the division unit, and the output unit are provided on the same device. It was configured as 15. The information processing system according to claim 14.

17. The information processing system A control step of controlling a vertical blanking period of an image displayed on the display unit that emits light by a passive matrix driving method based on an exposure time of an image capturing unit that captures the display unit by a rolling shutter method. Including, In the control step, the vertical blanking period is controlled so that the exposure time is an integer multiple of a light emission period of the display unit. Information processing methods.

18. In the control step, the vertical blanking period of a first frame, which is one of two adjacent frames of the image, is controlled to be deleted based on the exposure time.

18. The information processing method according to claim 17.

Images