Control device, image capture system and 3D model data generation method

Abstract

The present technology relates to a control device, an image capture system, and a 3D model data generation method for enabling appropriate extraction of a subject from a moving image. The control device according to the present technology includes: a control unit that performs control to synchronize the image capture timing of an image capture device that performs image capture with the display timing of a display device located in the image capture range of the image capture device to display a first moving image; and an image processing unit that performs foreground-background difference processing based on a second moving image captured by the image capture device at a first timing and a third moving image captured by the image capture device at a second timing different from the first timing. The second moving image and the third moving image each include at least a portion of the first moving image whose display timing is synchronized between the second moving image and the third moving image. The present technology is applicable to, for example, an image capture system for capturing moving images used in 3D model generation.

Description

Technical Field

[0001] The present technology relates to a control device, an imaging system, and a 3D model data generation method, and more particularly, to a control device, an imaging system, and a 3D model data generation method capable of appropriately extracting a subject from a moving image. Background Art

[0002] There is a technology that generates a 3D model of a subject from moving images captured from multiple viewpoints and generates virtual viewpoint images of the 3D model from arbitrary viewpoint positions to provide images from free viewpoints. Such technology is also called volume capture technology or the like.

[0003] For example, Patent Document 1 describes a technique for generating a 3D model using a method such as Visual Hull in which the three-dimensional shape of a subject is cut out based on a plurality of captured images obtained by imaging from different directions.

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: International Patent Application Publication No. 2018 / 150933 Summary of the invention

[0007] Problems to be solved by the present invention

[0008] In the generation of a 3D model, it is necessary to separate the subject and the background that appear in each of the captured images captured from multiple viewpoints. For example, by taking the difference between a background image obtained by imaging only the background in advance and a captured image obtained by imaging in a state where a subject (such as a person, etc.) actually exists, the subject and the background that appear in each captured image are separated.

[0009] Incidentally, in imaging for generating a 3D model, a case can be conceived in which a person as a subject displays a moving image on a display device to confirm the content of an action, or displays lines and lyrics on a prompter for reference.

[0010] However, in actual imaging in a state where a person exists, if there is an object in the imaging space that does not exist in the background imaging, the object itself is also regarded as a subject. For example, in the case where a moving image is displayed on a display device arranged in the imaging space, the display of the moving image is also separated as a subject together with the person, and is used to generate a 3D model. Therefore, in actual imaging, it is difficult for the person as the subject to check the content of the action with reference to the display of the display device, or it is difficult to check the lines or lyrics with reference to the display of the prompter.

[0011] The present technology has been proposed in view of such circumstances, and aims to appropriately extract a subject from a moving image.

[0012] Solution to the problem

[0013] A control device according to one aspect of the present technology is a control device comprising: a control unit that performs control to synchronize the imaging timing of an imaging device performing imaging with the display timing of a display device located in an imaging range of the imaging device displaying a first motion image; and an image processing unit that performs foreground-background difference processing based on a second motion image captured by the imaging device at a first timing and a third motion image captured by the imaging device at a second timing different from the first timing, wherein both the second motion image and the third motion image include at least a portion of the first motion image whose display timing is synchronized between the second motion image and the third motion image.

[0014] An imaging system according to one aspect of the present technology is an imaging system comprising: a plurality of imaging devices; at least one display device; a control unit that performs control to synchronize imaging timing of the plurality of imaging devices performing imaging with display timing of a first motion image displayed by a display device located in an imaging range of at least one of the plurality of imaging devices; and an image processing unit that performs foreground-background difference processing based on a second motion image captured by the imaging device at a first timing and a third motion image captured by the imaging device at a second timing different from the first timing, wherein both the second motion image and the third motion image include at least a portion of the first motion image.

[0015] A 3D model data generation method according to one aspect of the present technology is the following 3D model data generation method, which includes: synchronizing the imaging timing of multiple imaging devices performing imaging with the display timing of a display device located in the imaging range of at least one imaging device among the multiple imaging devices to display a first motion image, and performing imaging by the multiple imaging devices; generating differential data by performing foreground-background differential processing based on a second motion image captured by the imaging device at a first timing and a third motion image captured by the imaging device at a second timing different from the first timing; and generating 3D model data based on the differential data.

[0016] In a control device according to one aspect of the present technology, control is performed to synchronize imaging timing at which an imaging device performs imaging with display timing at which a display device located in an imaging range of the imaging device displays a first moving image, and to perform foreground-background difference processing based on a second moving image captured by the imaging device at the first timing and a third moving image captured by the imaging device at a second timing different from the first timing. Both the second moving image and the third moving image include at least a portion of the first moving image, and the display timing of the first moving image is synchronized between the second moving image and the third moving image.

[0017] In an imaging system according to one aspect of the present technology, control is performed to synchronize imaging timings at which a plurality of imaging devices perform imaging with display timings at which a display device located in an imaging range of at least one of the plurality of imaging devices displays a first moving image, and foreground-background difference processing is performed based on a second moving image captured by the imaging device at the first timing and a third moving image captured by the imaging device at a second timing different from the first timing. Both the second moving image and the third moving image include at least a portion of the first moving image.

[0018] In a 3D model data generation method according to one aspect of the present technology, imaging timings of multiple imaging devices performing imaging are synchronized with display timings of a first motion image displayed by a display device located in an imaging range of at least one of the multiple imaging devices, and imaging is performed by the multiple imaging devices, differential data is generated by performing foreground-background differential processing based on a second motion image captured by the imaging device at a first timing and a third motion image captured by the imaging device at a second timing different from the first timing, and 3D model data is generated based on the differential data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a diagram showing a configuration example of an imaging system to which the present technology is applied.

[0020] Figure 2 FIG. 1 is a diagram showing an overview of 3D model generation executed by the control device.

[0021] Figure 3 is a plan view showing an example of a case when the imaging system performs background imaging with N=8.

[0022] Figure 4 is a plan view showing an example of a situation when the imaging system performs actual imaging with N=8.

[0023] Figure 5 is a diagram showing an example of control of imaging timing and display timing based on a synchronization signal.

[0024] Figure 6 is a diagram illustrating an example of foreground-background difference processing.

[0025] Figure 7 is a block diagram showing a configuration example of an imaging system.

[0026] Figure 8 : is a flowchart showing the synchronized moving image generation process of the control device.

[0027] Fig. 9 is a flowchart illustrating a moving image capturing process.

[0028] Fig.10 : is a flowchart showing the object 3D model data generation process executed by the control device.

[0029] Fig.11 is a flowchart showing the details of the foreground-background difference processing.

[0030] Fig.12 is a block diagram showing a configuration example of an imaging system in the case of performing monitor display.

[0031] Fig.13 1 is a flowchart showing a synchronized moving image generation process of the control device when executing monitor display.

[0032] Fig.14 is a diagram showing another configuration example of the imaging system.

[0033] Fig.15 It is shown in Fig.14 A plan view of the actual imaging situation when N=8 is used in the imaging system in .

[0034] Fig.16 is a diagram showing still another configuration example of the imaging system.

[0035] Fig.17 is a block diagram showing a hardware configuration example of a computer. DETAILED DESCRIPTION

[0036] Hereinafter, a mode for carrying out the present technology will be described. The description will be given in the following order.

[0037] 1. Overview of the Imaging System

[0038] 2. Configuration of the Imaging System

[0039] 3. Operation of the Imaging System

[0040] 4. Processing when executing monitoring display

[0041] 5. Modifications

[0042] 6. Example of computer configuration.

[0043] <1. Overview of Imaging System>

[0044] Figure 1 is a diagram showing a configuration example of an imaging system 1 to which the present technology is applied.

[0045] Figure 1 The imaging system 1 in EMBODIMENT 1 includes a control device 11, N (N>1) imaging devices 12-1 to 12-N, and a monitor 13. The imaging devices 12-1 to 12-N and the monitor 13 are arranged in the same space (eg, an imaging studio).

[0046] The control device 11 includes, for example, a computer, a server device, etc., and controls imaging timing at which the camera devices 12 - 1 to 12 -N perform imaging and display timing at which the monitor 13 displays a moving image.

[0047] Specifically, the control device 11 generates a synchronization signal and supplies the synchronization signal to each of the camera devices 12-1 to 12-N and the monitor 13. The imaging timing at which the camera devices 12-1 to 12-N perform imaging and the display timing at which the monitor 13 displays a moving image are controlled based on the synchronization signal supplied from the control device 11. In addition, the control device 11 also reproduces the moving image displayed on the monitor 13.

[0048] Furthermore, the control device 11 generates, for example, a 3D model of an object in which a person appearing as a subject is the object, based on a plurality of captured moving images supplied from each of the imaging devices 12 - 1 to 12 -N.

[0049] The data of the 3D model of the object includes, for example, motion image data of the object generated from captured motion images obtained by the cameras 12-1 to 12-N and 3D shape data representing the 3D shape of the object. The 3D shape data is represented by, for example, a depth image corresponding to the captured motion images captured by the plurality of cameras 12-1 to 12-N, a point cloud representing the three-dimensional position of the object by a set of points, a polygon mesh represented by connections between vertices, and the like.

[0050] The camera devices 12-1 to 12-N are imaging devices that capture a subject to be extracted as an object and its background under the control of the control device 11. The camera devices 12-1 to 12-N provide the captured motion images obtained by imaging to the control device 11. For example, the camera devices 12-1 to 12-N are arranged around the subject to image the same subject. The captured motion images provided from the camera devices 12-1 to 12-N are multi-viewpoint motion images, which include a plurality of captured motion images having different viewpoints for capturing the subject.

[0051] In the following description, in the case where it is not necessary to distinguish between the imaging devices 12-1 to 12-N, the imaging device is simply referred to as the imaging device 12. The same applies to other configurations provided in a plurality of units.

[0052] The monitor 13 is a display device that displays a reference moving image as a first moving image under the control of the control device 11. The reference moving image is a moving image to be referred to by a person who is a subject at the time of actual imaging. For example, at the time of actual imaging, a moving image serving as a model of the action of a person who is a subject is displayed on the monitor 13 as a reference moving image.

[0053] In addition, the monitor 13 can also perform monitoring display in which a monitoring image is displayed as a reference motion image. The monitoring image is a captured motion image captured in real time by any one of the camera devices 12-1 to 12-N. In the monitoring display, a person as a subject can perform a predetermined action while confirming his or her own motion.

[0054] In addition, the monitor 13 can also display a moving image showing the lines and lyrics as a reference moving image. By displaying a moving image showing the lines and lyrics, the person being the subject can perform while checking the lines, or sing while checking the lyrics.

[0055] Note that the monitor 13 may be configured by a device such as a smartphone or a tablet terminal (portable terminal) including a display.

[0056] Communication between the control device 11 and the cameras 12-1 to 12-N and between the control device 11 and the monitor 13 may be performed directly via a cable or the like, or may be performed via a predetermined network such as a local area network (LAN) or the Internet. Furthermore, communication may be wired communication or wireless communication.

[0057] Next, we will refer to Figure 2 An outline of 3D model generation performed by the control device 11 is described.

[0058] Figure 2 An example is shown in which a person performing a predetermined operation is imaged as the subject Ob1 using three imaging devices 12. Figure 2 As shown on the left side of FIG. 1 , three imaging devices 12 - 1 to 12 - 3 arranged around the object Ob1 image the object Ob1.

[0059] 3D modeling is performed by the control device 11 using captured moving images obtained from a plurality of camera devices 12 arranged at different positions, and as shown in FIG. Figure 2As shown in the center of , a 3D model MO1 of the subject Ob1 is generated. For example, the 3D model MO1 is generated by a method such as a visual shell in which a three-dimensional shape is cut using captured moving images obtained by imaging the subject Ob1 from different directions.

[0060] The data of the 3D model of the subject generated as described above (3D model data) is transmitted to the device on the reproduction side and reproduced. That is, in the device on the reproduction side, a 3D model is presented based on the 3D model data, thereby displaying a 3D shape video on the viewing device. Figure 2 , a display D1 and a head mounted display (HMD) D2 are shown as viewing devices used by a viewer.

[0061] Incidentally, in the generation of the 3D model, it is necessary to separate the subject and the background appearing in each captured moving image, and extract only the data of the subject. As a method of extracting only the data of the subject, keying using a green screen background is known. In addition, the subject and the background can be separated by obtaining the difference between a background moving image, which is a moving image in which only the background is imaged, and an actual moving image, which is a moving image in which the subject and the background are imaged, through foreground-background difference processing.

[0062] Figure 1 The imaging system 1 in the present invention performs background imaging as imaging of a background moving image and actual imaging as imaging of an actual moving image, and obtains the difference between the background moving image and the actual moving image through foreground-background difference processing, thereby separating the subject and the background. Here, the actual imaging is imaging in the presence of a subject, and the background imaging is imaging in the absence of a subject. The actual moving image is a second moving image captured by the camera 12 at a predetermined first timing (timing of actual imaging), and the background moving image is a third moving image captured by the camera 12 at a second timing (timing of background imaging) different from the first timing.

[0063] Figure 3 is a plan view showing an example of a case when the imaging system 1 performs background imaging with N=8.

[0064] exist Figure 3 In the example of , eight cameras 12-1 to 12-8 are arranged in a ring at substantially equal intervals to face the central portion of an imaging space (e.g., a room). Camera 12-1 and camera 12-5, camera 12-2 and camera 12-6, camera 12-3 and camera 12-7, and camera 12-4 and camera 12-8 are arranged to face each other.

[0065] Monitor 13 is arranged between camera 12-5 and camera 12-6, with the front of monitor 13 facing the center. Here, monitor 13 is located within the imaging range of camera 12-2. As shown in balloon #1, a moving image of a person dancing is displayed on monitor 13 as a reference moving image.

[0066] In this way, background imaging is performed in a state where no subject exists. In the background motion image captured by each of the cameras 12-1 to 12-8, another camera 12, the monitor 13, the wall, ceiling, floor, etc. of the imaging space present in the viewing angle are represented as the background.

[0067] For example, as shown in the balloon #2, in the background moving image captured by the camera 12-2, the camera 12-6 appears near the center, and the monitor 13 appears on the left side of the camera 12-6. Since the reference moving image is displayed on the monitor 13 when the background is imaged, the reference moving image showing a state in which a person is dancing is also displayed on the monitor 13 appearing in the background moving image captured by the camera 12-2.

[0068] Figure 4 is a plan view showing an example of a situation when the imaging system 1 performs actual imaging with N=8.

[0069] like Figure 4 As shown, the arrangement of the cameras 12-1 to 12-8 and the monitor 13 during actual imaging is similar to that of the reference image. Figure 3 The arrangement described for background imaging is similar.

[0070] Actual imaging is performed with a person as the subject Ob11 existing in the center portion of the imaging space. In the actual moving image captured by each of the camera devices 12-1 to 12-8, the subject Ob11 existing in the angle of view and the background appears. Note that at the time of actual imaging, as shown in the balloon #11, the same moving image as that displayed at the time of background imaging is displayed on the monitor 13 as a reference moving image.

[0071] As shown in the balloon #12, in the actual moving image captured by the camera 12-2, the subject Ob11 appears near the center, and the monitor 13 appears at a position on the left rear side of the subject Ob11. Since the reference moving image is displayed on the monitor 13 at the time of actual imaging, the reference moving image showing a state in which a person is dancing is also displayed on the monitor 13 that appears in the actual moving image captured by the camera 12-2.

[0072] As described above, actual imaging is performed similarly to performing background imaging, except that imaging is performed by arranging the subject in the center portion of the imaging space.

[0073] In the case where the subject is separated from the background by foreground-background difference processing, if an object that does not exist when the background is imaged is present in the imaging space when it is actually imaged, the object itself is also regarded as the subject.

[0074] For example, with respect to the actual imaging time and the background imaging time, if the reference motion image of the monitor 13 appearing in the actual motion image captured by the camera 12-2 is different from the reference motion image displayed on the monitor 13 during the background imaging, the reference motion image of the monitor 13 appearing in the actual motion image is also extracted as the subject.

[0075] In this context, the control device 11 of the imaging system 1 generates a synchronization signal and supplies the synchronization signal to the cameras 12-1 to 12-8 and the monitor 13, thereby performing control so that the imaging timing of the cameras 12-1 to 12-8 and the display timing of the monitor 13 are synchronized with each other.

[0076] Figure 5 is a diagram showing an example of control of imaging timing and display timing based on a synchronization signal.

[0077] For example, at each imaging time in background imaging and actual imaging, Figure 5 A synchronization signal shown on the upper side is supplied to each of the camera 12 and the monitor 13 . Figure 5 The sync signal shown is a pulse signal that alternately repeats 0 (low) and 1 (high) in a predetermined cycle. Figure 5 In the example of , the synchronization signal rises at time t1, time t3, time t5, time t7, ..., and falls at time t2, time t4, time t6, time t8, ...

[0078] The image pickup device 12 performs exposure (imaging) for acquiring a frame image included in a moving image (actual moving image or background moving image) at each timing of times t1 , t3 , t5 , and t7 which are rising times of the synchronization signal.

[0079] Furthermore, the monitor 13 sequentially displays one or more frame images included in the reference moving image at each timing of times t1, t3, t5, and t7 as the rising time of the synchronization signal. As described above, the reference moving image is displayed during each of actual imaging and background imaging.

[0080] Note that the control device 11 decodes the frame image before the update timing (display timing) of the frame image displayed on the monitor 13. For example, Figure 5 As shown, the frame image displayed at the timing of time t1 is decoded before time t1. Similarly, the frame image displayed at each timing of time t3, t5, and t7 is decoded before each timing.

[0081] At each of the actual imaging time and the background imaging time, the control device 11 generates a synchronization signal and provides the synchronization signal to the cameras 12-1 to 12-8 and the monitor 13, so that at least a portion of the same frame image of the reference motion image appears in the frame images of the same frame number of the actual motion image and the background motion image.

[0082] Since the same frame image of the reference moving image is included in the frame images of the same frame number of the actual moving image and the background moving image, when the subject and the background are separated by the foreground-background difference processing, the area of ​​the reference moving image displayed on the monitor 13 in the actual moving image can be regarded as the background. Therefore, it is possible to prevent the reference moving image on the monitor 13 appearing in the actual moving image from being extracted as the subject. Therefore, the imaging system 1 can perform imaging while providing the reference moving image to a person as the subject in the imaging for 3D model generation.

[0083] Note that although in the above example the imaging by the camera 12 and the display of the reference motion image by the monitor 13 are performed at the rising timing of the synchronization signal, the imaging by the camera 12 and the display of the reference motion image by the monitor 13 may also be performed at the falling timing of the synchronization signal.

[0084] Figure 6 is a diagram showing an example of foreground-background difference processing.

[0085] exist Figure 6 The actual motion image shown on the left is a reference Figure 4 The actual moving image captured by the camera 12-2 is described. Figure 6 The background moving image shown in the center is a reference Figure 3 Described is a background moving image captured by the camera 12 - 2 .

[0086] By obtaining the difference between the actual motion image and the background motion image, such as Figure 6 As shown on the right side of , the control device 11 extracts data of a region in which the object Ob11 appears as differential data.

[0087] Specifically, the control device 11 compares the pixel values ​​of the pixels included in the actual moving image with the pixels of the background moving image at the same pixel positions as the pixels included in the actual moving image, and extracts the pixels in the actual moving image in which the difference between the pixel values ​​is equal to or greater than a predetermined threshold as the pixels of the foreground. Such a comparison of the pixel values ​​is performed on all the pixels included in the actual moving image, and the pixels of the foreground (i.e., the pixels included in the subject) are generated as differential data.

[0088] Comparison of pixel values ​​can be performed by comparing RGB values, or can be performed by comparing HSV (Hue, Saturation, Value) values ​​obtained by converting RGB values. In the comparison using HSV values, control can be performed so that a change in pixel value in which only the brightness changes but the color (hue) does not change (such as the presence or absence of a shadow) is not extracted as a change. In addition, differential data can be generated by comparing not only the pixel value of a pixel to be compared but also the pixel values ​​of pixels around the pixel to be compared. The threshold for determining the pixel of the foreground can be appropriately set according to the imaging environment.

[0089] The foreground-background difference processing as described above is performed on actual moving images and background moving images captured by a plurality of camera devices 12 , and difference data of the moving image is generated for each camera device 12 .

[0090] As described above, according to the imaging system 1, a subject can be appropriately extracted from an actual moving image in which the monitor 13 displaying the reference moving image appears. Even if the image displayed on the monitor 13 changes during imaging, foreground-background difference processing can be performed without being affected by the change.

[0091] <2. Configuration of imaging system>

[0092] Figure 7 is a block diagram showing a configuration example of the imaging system 1 .

[0093] The imaging system 1 includes a control device 11, N imaging devices 12-n (n=1 to N), and a monitor 13. The imaging device 12-n is provided with an imaging unit 51. Furthermore, the monitor 13 is provided with a display unit 52.

[0094] The control device 11 includes an operation unit 31 , a control unit 32 , a reproduction unit 33 , a synchronization signal generation unit 34 , a storage control unit 35 , a storage unit 36 ​​, and an image processing unit 37 .

[0095] The operation unit 31 includes a touch panel monitor, keyboard, mouse, controller, remote operation device, etc. The operation unit 31 detects an operation performed by an image capturing person or a person as a subject, and supplies information indicating the content of the detected operation to the control unit 32 .

[0096] For example, the operation unit 31 detects an operation for starting background imaging, an operation for starting actual imaging, an operation for generating 3D model data, etc. An operation for specifying a reference moving image displayed on the monitor 13 at the time of background imaging and at the time of actual imaging can be detected by the operation unit 31.

[0097] The control unit 32 controls each configuration of the control device 11 and the imaging device 12 - n based on information supplied from the operation unit 31 .

[0098] For example, in the case where an operation for starting actual imaging or background imaging is detected by the operation unit 31, the control unit 32 controls the imaging unit 51 of the camera 12-n, and performs various settings such as the setting of the frame rate and the setting of the resolution of the moving image. In addition, the control unit 32 performs control so that the reproduction unit 33 displays a predetermined reference moving image on the monitor 13. In addition, the control unit 32 performs control so that the synchronization signal generation unit 34 generates a synchronization signal.

[0099] On the other hand, in the case where the operation unit 31 detects an operation for generating 3D model data, the control unit 32 performs control to cause the image processing unit 37 to perform 3D model data generation processing.

[0100] The reproducing unit 33 acquires and reproduces the reference moving image specified by the control unit 32 from the reference moving images stored in the storage unit 36. The reproducing unit 33 decodes the reference moving image before the display timing of the monitor 13, and supplies the reference moving image obtained by the decoding to the display unit 52. The decoding of the reference moving image is performed by a method corresponding to the encoding method used when the reference moving image is stored in the storage unit 36. Note that it is assumed that the frame rate of the reference moving image is the same as the frame rate when the imaging unit 51 performs actual imaging or performs background imaging.

[0101] The synchronization signal generating unit 34 generates a reference signal under the control of the control unit 32. Figure 5 The described synchronization signal is generated, and the synchronization signal is supplied to each of the imaging unit 51 and the display unit 52.

[0102] The storage control unit 35 encodes the moving image supplied from the imaging unit 51 by a predetermined encoding method (for example, the MPEG2 method or the AVC method), and stores the encoded moving image in the storage unit 36 ​​.

[0103] The storage unit 36 ​​includes an auxiliary storage device including an internal memory or an external memory (eg, a semiconductor memory). The storage unit 36 ​​includes a reproduction data storage unit 41 and an imaging data storage unit 42.

[0104] The reproduction data storage unit 41 stores the reference moving image reproduced by the reproduction unit 33 .

[0105] The imaging data storage unit 42 stores the background moving image and the actual moving image supplied from the storage control unit 35. The imaging data storage unit 42 also stores information about the camera 12-n used for performing moving image capturing together with the background moving image and the actual moving image. The information about the camera 12-n includes, for example, internal parameters and external parameters of the camera 12-n.

[0106] Note that the reproduction data storage unit 41 and the imaging data storage unit 42 may be provided as separate storage units in the control device 11 .

[0107] The image processing unit 37 generates 3D model data of the object under the control of the control unit 32. The image processing unit 37 includes a foreground-background difference processing unit 43 and a 3D model data generating unit 44.

[0108] The foreground-background difference processing unit 43 acquires the actual moving image and the background moving image of each camera 12 from the storage unit 36. The foreground-background difference processing unit 43 performs foreground-background difference processing using the actual moving image and the background moving image to generate difference data. The difference data generated by the foreground-background difference processing unit 43 is provided to the 3D model data generating unit 44.

[0109] The 3D model data generating unit 44 acquires information about each camera 12 from the storage unit 36. The 3D model data generating unit 44 generates 3D model data of the subject using a method such as a visual shell based on the difference data of each camera 12 supplied from the foreground-background difference processing unit 43 and the information about each camera 12. The 3D model data of the subject generated by the 3D model data generating unit 44 is stored in the storage unit 36 ​​or output to an external device.

[0110] Note that the image processing unit 37 may be provided not as a part of the control device 11 but as an image processing device different from the control device 11 .

[0111] The imaging unit 51 of the camera 12-n captures an actual moving image or a background moving image at an imaging timing based on the synchronization signal supplied from the synchronization signal generating unit 34. The moving image obtained by capturing the actual moving image or the background moving image is supplied to the storage control unit 35 of the control device 11.

[0112] The display unit 52 of the monitor 13 displays the reference moving image supplied from the reproduction unit 33 at display timing based on the synchronization signal supplied from the synchronization signal generation unit 34 .

[0113] <3. Operation of Imaging System>

[0114] Next, we will refer to Figures 8 to 11 The operation of the imaging system 1 is described with reference to a flowchart.

[0115] First, refer to Figure 8 The synchronized motion image generation process of the control device 11 is described with reference to the flowchart of FIG.

[0116] Figure 8 The synchronized moving image generation process in is a series of processes for capturing a background moving image and an actual moving image. For example, when an operation for starting background imaging is detected by the operation unit 31, the process starts.

[0117] In step S1, the control device 11 executes a background moving image capturing process. The background moving image capturing process is executed in a state where no subject exists in the imaging space. Fig. 9 A background moving image is generated by the background moving image capturing process and is stored in the storage unit 36.

[0118] Next, when an operation for starting actual imaging is detected by the operation unit 31, the control device 11 executes actual moving image capturing processing in step S2. The actual moving image capturing processing is to execute the actual moving image capturing processing described later in a state where a subject exists in the imaging space. Fig. 9 The actual moving image is generated by the actual moving image capturing process, and the actual moving image is stored in the storage unit 36. When the background moving image and the actual moving image are stored in the storage unit 36, the synchronized moving image generation process ends.

[0119] Next, we will refer to Fig. 9 The flowchart is used to describe the Figure 8 The moving image capturing process of the process in step S1 or step S2.

[0120] In step S11, the control unit 32 prepares the camera 12 and the monitor 13. Specifically, the control unit 32 controls the imaging unit 51 to perform various settings, such as the setting of the frame rate and the setting of the resolution of the moving image. In addition, the control unit 32 performs control so that the reproduction unit 33 displays the reference moving image on the monitor 13. The reproduction unit 33 acquires the reference moving image specified by the control unit 32, decodes the reference moving image, and appropriately provides the frame image of the reference moving image to the display unit 52.

[0121] In step S12 , the control unit 32 causes the synchronization signal generation unit 34 to start generating a synchronization signal. Under the control of the control unit 32 , the synchronization signal generation unit 34 generates a synchronization signal and supplies the synchronization signal to the imaging unit 51 and the display unit 52 of each camera 12 .

[0122] In step S13, the imaging unit 51 of each camera 12 captures a background moving image or an actual moving image based on the synchronization signal supplied from the synchronization signal generating unit 34. At the same time, the display unit 52 displays a reference moving image based on the synchronization signal supplied from the synchronization signal generating unit 34.

[0123] In step S14, the control unit 32 causes the synchronization signal generating unit 34 to stop generating the synchronization signal. For example, the process of step S14 is performed when an operation for ending imaging is detected by the operating unit 31. The flow of the process may be configured such that the process of step S14 is performed when a predetermined time has passed from the generation of the synchronization signal.

[0124] In step S15 , the control unit 32 receives the imaging end notification supplied from the imaging unit 51 .

[0125] In step S16, the storage control unit 35 stores the moving image captured by each camera 12 in the storage unit 36. That is, the storage control unit 35 encodes the background moving image or the actual moving image captured and provided by the imaging unit 51 in the process of step S13 by a predetermined encoding method (for example, the MPEG2 method or the AVC method), and stores the encoded moving image in the storage unit 36.

[0126] Therefore, as Figure 8 The motion image capture processing performed by the processing of step S1 or step S2 in is completed. Figure 8 The difference between the background moving image capturing process of step S1 and the actual moving image capturing process of step S2 is whether the process is performed in a state where a subject exists in the imaging space.

[0127] Next, we will refer to Fig.10The flowchart of 2000 describes the subject 3D model data generation processing performed by the control device 11. For example, when an operation for generating 3D model data of a subject is detected by the operation unit 31, the processing starts.

[0128] First, in step S31, the foreground-background difference processing unit 43 performs foreground-background difference processing on each actual moving image captured by each camera 12. Differential data of the moving image of each camera 12 is generated by the foreground-background difference processing. Fig.11 The flowchart describes the details of the foreground-background difference processing.

[0129] In step S32 , the 3D model data generating unit 44 generates 3D model data of the object based on the difference data of the moving image of each camera 12 .

[0130] Will refer to Fig.11 The flowchart is described in Fig.10 Details of the foreground-background difference processing performed in step S31.

[0131] In step S41, the foreground-background difference processing unit 43 reads a predetermined frame image from a plurality of frame images included in the actual motion image from the storage unit 36. For example, in the process of step S41 repeatedly performed as described below, the foreground-background difference processing unit 43 sequentially reads a plurality of frame images included in the actual motion image frame by frame from the first frame image.

[0132] In step S42, the foreground-background difference processing unit 43 reads, from the storage unit 36, a frame image of the background motion image having the same frame number as the frame image of the actual motion image read in step S41, among a plurality of frame images included in the background motion image.

[0133] In step S43, the foreground-background difference processing unit 43 sets a predetermined pixel among a plurality of pixels included in the frame image of the actual moving image read from the storage unit 36 ​​as a processing target pixel, and calculates the difference in pixel values ​​between the processing target pixel and a pixel corresponding to the processing target pixel among pixels included in the frame image of the background moving image. Then, the foreground-background difference processing unit 43 determines whether the difference between the calculated pixel values ​​is equal to or greater than a predetermined threshold value.

[0134] If it is determined in step S43 that the difference between the pixel values ​​is equal to or greater than the predetermined threshold, the process proceeds to step S44, and the foreground-background difference processing unit 43 sets the processing target pixel as a foreground pixel.

[0135] On the other hand, if it is determined in step S43 that the difference between the pixel values ​​is smaller than the predetermined threshold, the process proceeds to step S45, and the foreground-background difference processing unit 43 sets the processing target pixel as the background pixel.

[0136] In step S46, the foreground-background difference processing unit 43 determines whether there is a next pixel to be set as a processing target pixel among the pixels of the frame image of the actual moving image read in step S41. In step S46, in the case where all the pixels of the frame image of the actual moving image read in step S41 are set as processing target pixels, it is determined that there is no next pixel to be set as the processing target pixel, and in the case where all the pixels have not yet been set as processing target pixels, it is determined that there is a next pixel to be set as the processing target pixel.

[0137] If it is determined in step S46 that the next pixel exists, the process returns to step S43, a pixel that has not yet been set as a processing target pixel is set as a processing target pixel, and subsequent processing is performed.

[0138] On the other hand, if it is determined in step S46 that there is no next pixel, the process proceeds to step S47, and the foreground-background difference processing unit 43 generates only a frame image of the foreground. That is, in step S47, a frame image is generated in which only pixels set as foreground pixels are collected in the frame image of the actual motion image read in step S41. For example, a fixed pixel value (e.g., 0) is stored in the pixel set as the background pixel.

[0139] Next, in step S48, the foreground-background difference processing unit 43 determines whether there is a next frame image, that is, whether there is a frame image for which generation of only the foreground has not yet been performed in the actual moving image.

[0140] If it is determined in step S48 that there is a next frame image, the process returns to step S41, the frame image of the actual moving image for which generation of only the frame image of the foreground has not yet been performed is read as the next frame image, and subsequent processing is performed.

[0141] On the other hand, if it is determined in step S48 that there is no next frame image, the process proceeds to step S49, and the foreground-background difference processing unit 43 generates difference data of the moving image. That is, the foreground-background difference processing unit 43 generates difference data of the moving image by arranging the frame images of only the foreground generated by the process repeatedly performed in step S47 in the same order as the frame images of the actual moving image.

[0142] therefore, Fig.11 The foreground-background difference processing is completed. Fig.11The foreground-background difference processing in is performed on each of the actual moving images captured by the plurality of camera devices 12. The foreground-background difference processing may be performed by sequentially selecting the actual moving images corresponding to the camera devices 12 one by one, or the foreground-background difference processing may be performed by selecting two or more or all of the actual moving images at the same time. After generating the difference data of the moving images for all the actual moving images corresponding to all the camera devices 12, the processing returns to Fig.10 The process of step S31 and subsequent steps is performed.

[0143] According to the above-described imaging system 1, a person as a subject can perform imaging while confirming a reference moving image displayed on the monitor 13 at the time of actual imaging. At this time, it is possible to prevent the moving image displayed on the monitor 13 from being extracted as a subject.

[0144] <4. Processing when executing monitoring display>

[0145] Next, a case where the imaging system 1 captures a moving image while performing monitor display will be described.

[0146] When the imaging system 1 performs monitoring display, as described above, the moving image captured by the camera 12 is displayed in real time as a reference moving image on the monitor 13. That is, at the time of actual imaging, the captured actual moving image is displayed as is on the monitor 13 as a reference moving image.

[0147] In this case, since it is necessary to display the actual moving image as the reference moving image on the monitor 13 at the time of background imaging, the actual imaging is performed before the background imaging, and the background imaging is performed after the actual imaging. At the time of background imaging, the actual moving image displayed on the monitor 13 at the time of actual imaging and stored in the control device 11 is displayed as the reference moving image.

[0148] <Configuration of Imaging System 1>

[0149] Fig.12 is a block diagram showing a configuration example of the imaging system 1 in the case where monitor display is performed.

[0150] exist Fig.12 In, with Figure 7 Configuration of the imaging system 1 in FIG. 1A and FIG. 1B . The same configurations are denoted by the same reference numerals, and redundant description will be appropriately omitted.

[0151] Fig.12 The configuration of the imaging system 1 shown is similar to that of Figure 7 The configuration of the imaging system 1 described above is different in that an actual moving image obtained by the camera 12 - m is provided to the display unit 52 .

[0152] The camera 12-m (m is any one of 1 to N) is a camera that captures an actual moving image as a reference moving image displayed on the monitor 13. The camera 12-m is supplied with a synchronization signal that is the same as the synchronization signal of the camera 12-n (n is 1 to N excluding m) from the synchronization signal generating unit 34.

[0153] Similar to the camera 12-n, the camera 12-m is provided with an imaging unit 51-m. The imaging unit 51-m captures an actual moving image or a background moving image at an imaging timing based on a synchronization signal provided from the synchronization signal generating unit 34. At the time of actual imaging, the actual moving image captured by the imaging unit 51-m is provided to the storage control unit 35 and the display unit 52. At the time of background imaging, the background moving image captured by the imaging unit 51-m is not provided to the display unit 52, but is provided to the storage control unit 35.

[0154] At the time of actual imaging, the display unit 52 displays the actual moving image supplied from the imaging unit 51-m as a reference moving image at the display timing based on the synchronization signal supplied from the synchronization signal generating unit 34. On the other hand, at the time of background imaging, the display unit 52 displays the reference moving image supplied from the reproduction unit 33 at the display timing based on the synchronization signal supplied from the synchronization signal generating unit 34. The reference moving image supplied from the reproduction unit 33 is the same as the actual moving image supplied from the imaging unit 51-m at the time of actual imaging.

[0155] The control unit 32 controls the imaging device 12-m similarly to the case of controlling the imaging device 12-n based on the information supplied from the operation unit 31. In the case where an operation for starting actual imaging or background imaging is detected by the operation unit 31, the control unit 32 controls the imaging unit 51-m and the imaging unit 51-n, and performs various settings such as the setting of the frame rate and the setting of the resolution of the moving image.

[0156] Furthermore, at the time of actual imaging, since the actual moving image captured by the imaging unit 51-m is displayed on the monitor 13 as a reference moving image, the control unit 32 does not cause the reproduction unit 33 to reproduce the predetermined moving image. In other words, the control unit 32 causes the reproduction unit 33 not to reproduce the predetermined moving image at the time of actual imaging. On the other hand, at the time of background imaging, the control unit 32 performs control so that the reproduction unit 33 displays again on the monitor 13 the actual moving image that was displayed on the monitor 13 at the time of actual imaging as a reference moving image.

[0157] At the time of actual imaging, the reproducing unit 33 does not reproduce a predetermined moving image under the control of the control unit 32. On the other hand, at the time of background imaging, the reproducing unit 33 acquires the actual moving image designated by the control unit 32 (the actual moving image captured by the imaging unit 51-m) from the actual moving images stored in the storage unit 36, and reproduces the acquired actual moving image. Figure 7 Similarly, the reproduction unit 33 decodes the reference moving image before the display timing of the monitor 13 , and supplies the actual moving image to the display unit 52 as the reference moving image.

[0158] The storage control unit 35 encodes the moving images supplied from the imaging units 51 - m and 51 - n by a predetermined encoding method, and stores the encoded moving images in the storage unit 36 ​​.

[0159] The reproduction data storage unit 41 stores the actual moving image captured by the imaging unit 51 - m as a reference moving image.

[0160] <Operation of Control Device 11>

[0161] Next, we will refer to Fig.13 The flowchart describes the synchronized moving image generation processing of the control device 11 in the case of performing monitor display.

[0162] For example, when an operation for starting actual imaging is detected by the operating unit 31, the Fig.13 Synchronous motion image generation processing in.

[0163] First, in step S101, the control device 11 performs actual moving image capturing processing. In the processing of step S101, Fig. 9 The moving image capturing process is performed in a state where the actual moving image captured by the camera 12-m is displayed on the monitor 13 and the subject exists in the imaging space. The actual moving image captured by each camera 12 through the actual moving image capturing process is stored in the storage unit 36.

[0164] Next, when an operation for starting background imaging is detected by the operation unit 31, the control device 11 performs background moving image capturing processing in step S102. In the processing of step S102, the actual moving image displayed on the monitor 13 in the processing of step S101 is displayed on the monitor 13 as a reference moving image, and Fig. 9 The moving image capturing process is performed in a state where no subject exists in the imaging space. Through the background moving image capturing process, the background moving image captured by each camera 12 is stored in the storage unit 36.

[0165] Use by Fig.13 The actual moving image and the background moving image generated by the synchronous moving image generation process in the above method are used to generate the 3D model data of the subject. Fig.10 The object 3D model data generation process shown is similar, so its description is omitted.

[0166] According to the above imaging system 1, while the person as the subject confirms his / her own movement during actual imaging, a moving image necessary for generating 3D model data can be generated. At this time, the moving image displayed on the monitor 13 can be prevented from being extracted as the subject.

[0167] <5. Modifications>

[0168] <Example of displaying time code>

[0169] At each of actual imaging and background imaging, the control unit 32 may perform control so that information indicating the time code is displayed on the monitor 13 together with the reference moving image.

[0170] In this case, it can be confirmed whether the time code of the moving image itself captured by the camera 12 matches the time code appearing in the moving image captured by the camera 12. Therefore, synchronization between the imaging timing of the camera 12 and the display timing of the monitor 13 can be easily confirmed.

[0171] <Example of Arranging Multiple Monitors 13>

[0172] There are cases where actual imaging is performed in a state where a plurality of persons as subjects exist, and where actual imaging is performed in a state where persons as subjects face various directions. In these cases, it is preferable to arrange a plurality of monitors 13 in the imaging space.

[0173] Fig.14 1 is another configuration example of the imaging system 1 , and is a diagram showing a configuration example of the imaging system 1 in a case where a plurality of monitors 13 are arranged in an imaging space.

[0174] Fig.14 The imaging system 1 in FIG. 1 includes a control device 11, N cameras 12-1 to 12-N, and N monitors 13-1 to 13-N. The cameras 12-1 to 12-N and the monitors 13-1 to 13-N are arranged in the same imaging space.

[0175] The monitors 13 - 1 to 13 -N may display different reference moving images, or may display the same reference moving image.

[0176] Even in the case where a plurality of monitors 13 are provided, the imaging timings of the cameras 12 - 1 to 12 -N and the display timings of the monitors 13 - 1 to 13 -N are controlled based on the synchronization signal generated by the control device 11 to synchronize them.

[0177] Fig.15 It is shown in Fig.14 A plan view of the situation when imaging is actually performed with N=8 in the imaging system 1.

[0178] exist Fig.15 In the example of FIG. 1 , eight cameras 12 - 1 to 12 - 8 and eight monitors 13 - 1 to 13 - 8 are alternately arranged in a ring shape.

[0179] Monitors 13-1 to 13-8 are arranged facing the center from the front side between camera 12-1 and camera 12-2, between camera 12-2 and camera 12-3, between camera 12-3 and camera 12-4, between camera 12-4 and camera 12-5, between camera 12-5 and camera 12-6, between camera 12-6 and camera 12-7, between camera 12-7 and camera 12-8, and between camera 12-8 and camera 12-1.

[0180] Note that Fig.14 and 15 In the example of FIG. 1 , the example in which the same number of monitors 13 as the number of camera devices 12 is provided in the imaging system 1 has been described. However, in the imaging system 1 , the number of monitors 13 may be different from the number of camera devices 12 .

[0181] As described above, even in the case where a plurality of monitors 13 are provided, the control device 11 can separate only a person assumed to be a subject as a foreground from the actual moving image by performing foreground-background difference processing using the actual moving image and the background moving image. At this time, the imaging timing of the camera devices 12-1 to 12-N and the display timing of the monitors 13-1 to 13-N are controlled based on the synchronization signal to synchronize them, thereby preventing the reference moving images on the monitors 13-1 to 13-N appearing in the actual moving image from being extracted as the subject.

[0182] <Example of Setting Synchronization Signal Generating Device>

[0183] Although the synchronization signal generating unit 34 is set Figure 7 The control device 11 in the imaging system 1 , however, the function of the synchronization signal generating unit 34 may be implemented by a device outside the control device 11 .

[0184] Fig.16is a diagram showing still another configuration example of the imaging system 1 , and is a diagram showing a configuration example of the imaging system 1 in a case where the function of the synchronization signal generating unit 34 is an independent device.

[0185] Fig.16 The illustrated imaging system 1 is configured by connecting a control device 11A, imaging devices 12 - 1 to 12 -N, a monitor 13 , and a synchronization signal generating device 101 directly or via a network.

[0186] The synchronization signal generating device 101 is provided with Figure 7 That is, the synchronization signal generating device 101 generates a synchronization signal under the control of the control device 11, and supplies the synchronization signal to each of the camera devices 12-1 to 12-N and the monitor 13. The control device 11A generates a synchronization signal under the control of the control device 11, and supplies the synchronization signal to each of the camera devices 12-1 to 12-N and the monitor 13. Figure 7 The configuration is performed by removing the synchronization signal generating unit 34 from the control device 11.

[0187] As described above, a part of the configuration provided in the control device 11 may be provided outside the control device 11 .

[0188] <Example of performing frame rate conversion>

[0189] In each of the above-described embodiments, the description has been given based on the assumption that the frame rate when the camera 12 performs actual imaging or background imaging is the same as the frame rate when the reproduction unit 33 reproduces the reference moving image. However, in some cases, the frame rate of the reference moving image may be different from the frame rate at which imaging is performed by the camera 12. In this case, the control unit 32 may specify the frame rate to the reproduction unit 33, and the reproduction unit 33 may convert the reference moving image specified from the control unit 32 into the specified frame rate, reproduce the reference moving image, and provide the reference moving image to the display unit 52.

[0190] In addition, for example, in the case where the lines or lyrics to be displayed as a reference moving image on the monitor 13 are not a moving image but include one or more still images, the reproduction unit 33 can convert the still image into a moving image having a frame rate specified by the control unit 32, and provide the moving image to the display unit 52.

[0191] <6. Example of computer configuration>

[0192] The above-described series of processing can be executed by hardware or software. In the case where the series of processing is executed by software, a program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, a general-purpose personal computer, or the like.

[0193] Fig.17: is a block diagram showing a hardware configuration example of a computer that executes the above-described series of processes according to a program.

[0194] A central processing unit (CPU) 301 , a read only memory (ROM) 302 , and a random access memory (RAM) 303 are connected to one another via a bus 304 .

[0195] An input / output interface 305 is also connected to the bus 304 . An input unit 306 , an output unit 307 , a storage unit 308 , a communication unit 309 , and a drive 310 are connected to the input / output interface 305 .

[0196] The input unit 306 includes a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 307 includes a display, a speaker, an output terminal, etc. The storage unit 308 includes a hard disk, a RAM disk, a nonvolatile memory, etc. The communication unit 309 includes a network interface, etc. The drive 310 drives a removable recording medium 311, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0197] In the computer configured as described above, the CPU 301 loads the program stored in the storage unit 308 onto the RAM 303 through the input / output interface 305 and the bus 304, and executes the program to perform, for example, the above-described series of processes. The RAM 303 also appropriately stores data necessary for the CPU 301 to perform various processes, etc.

[0198] The program executed by the computer (CPU 301) can be provided by being recorded on the removable recording medium 311 (eg, package medium). In addition, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0199] In the computer, by attaching the removable recording medium 311 to the drive 310, the program can be installed in the storage unit 308 through the input / output interface 305. In addition, the program can be received by the communication unit 309 through a wired or wireless transmission medium and installed in the storage unit 308. In addition, the program can be pre-installed in the ROM 302 or the storage unit 308.

[0200] Note that in this specification, the steps described in the flowchart can of course be performed in chronological order according to the described order, but do not necessarily have to be processed in chronological order, but can be performed in parallel or at necessary timing (for example, when called).

[0201] Note that in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), and it does not matter whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network and one device housing multiple modules in one housing are both systems.

[0202] Note that the effects described in this specification are merely illustrative and not restrictive. Therefore, other effects may be obtained.

[0203] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made without departing from the gist of the present technology.

[0204] For example, the present technology may have a cloud computing configuration in which one function is shared and processed by a plurality of devices through a network.

[0205] Furthermore, each step described in the above flowchart may be performed by one device or by a plurality of devices in a shared manner.

[0206] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in one step may be executed by one device, or may be executed by a plurality of devices in a shared manner.

[0207] <Exemplary Combinations of Configurations>

[0208] The present technology can also be configured as follows. (1)

[0210] A control device, comprising:

[0211] a control unit that performs control to synchronize an imaging timing at which an imaging device performs imaging with a display timing at which a display device located in an imaging range of the imaging device displays a first moving image, and

[0212] an image processing unit that performs foreground-background difference processing based on a second moving image captured by the imaging device at a first timing and a third moving image captured by the imaging device at a second timing different from the first timing, wherein

[0213] Both the second moving image and the third moving image include at least a portion of the first moving image whose display timing is synchronized between the second moving image and the third moving image. (2)

[0215] The control device according to (1) above, wherein:

[0216] The second moving image is an actual moving image, and the actual moving image is a moving image including a subject and a background;

[0217] The third moving image is a background moving image, and the background moving image is a moving image including the background; and

[0218] The image processing unit performs the foreground-background difference processing using the background moving image and the actual moving image. (3)

[0220] According to the control device of (2) above,

[0221] The first moving image is a reference moving image to be referred to when capturing the actual moving image, and

[0222] The control unit also performs control to cause the display device to display the reference moving image. (4)

[0224] According to the control device of (3) above,

[0225] The reference moving image is a moving image that is read from a predetermined storage unit and reproduced. (5)

[0227] The control device according to the above (4) further includes a reproduction unit that reproduces the reference moving image stored in the predetermined storage unit, wherein:

[0228] The reproduction unit decodes the reference moving image before a display timing of the display device. (6)

[0230] According to the control device of (3) above,

[0231] The reference moving image is a moving image captured by the imaging device when capturing the actual moving image. (7)

[0233] A control device according to any one of (3) to (6) above, wherein:

[0234] The control unit causes the display device to display the same reference moving image at the first timing and at the second timing. (8)

[0236] According to the control device of (7) above,

[0237] When imaging the actual moving image and the background moving image, the control unit causes the display device to display a frame image of the same reference moving image for the same frame number of the actual moving image and the background moving image. (9)

[0239] A control device according to any one of (1) to (8) above, wherein:

[0240] The control unit generates a synchronization signal, causes the imaging device to perform imaging at a rising timing or a falling timing of the synchronization signal, and causes the display device to display the first moving image. (10)

[0242] A control device according to any one of (1) to (9) above, wherein:

[0243] The control unit performs control to synchronize imaging timings at which the plurality of imaging devices perform imaging with display timings at which the plurality of display devices display the first moving image. (11)

[0245] The control device according to the above (10), wherein:

[0246] The control unit causes the plurality of display devices to respectively display different first moving images. (12)

[0248] A control device according to any one of (1) to (11) above, wherein:

[0249] The control unit also performs control to cause the display device to display information indicating a time code together with the first moving image. (13)

[0251] The control device according to the above (9) further comprises:

[0252] A synchronization signal generating unit generates the synchronization signal based on the control of the control unit. (14)

[0254] A control device according to any one of (1) to (13) above, wherein:

[0255] The image processing unit generates difference data by performing foreground-background difference processing between the second moving image and the third moving image captured by the imaging device, and further generates 3D model data based on the difference data. (15)

[0257] An imaging system, comprising:

[0258] multiple imaging devices;

[0259] at least one display device;

[0260] a control unit that performs control to synchronize imaging timings at which the plurality of imaging devices perform imaging with display timings at which a display device located in an imaging range of at least one of the plurality of imaging devices displays a first moving image; and

[0261] an image processing unit that performs foreground-background difference processing based on a second moving image captured by the imaging device at a first timing and a third moving image captured by the imaging device at a second timing different from the first timing, wherein

[0262] Both the second moving image and the third moving image include at least a portion of the first moving image. (16)

[0264] A method for generating 3D model data, comprising:

[0265] synchronizing imaging timings at which the plurality of imaging devices perform imaging with display timings at which a display device located in an imaging range of at least one of the plurality of imaging devices displays a first moving image, and performing imaging by the plurality of imaging devices;

[0266] generating difference data by performing foreground-background difference processing based on a second moving image captured by the imaging device at a first timing and a third moving image captured by the imaging device at a second timing different from the first timing; and

[0267] 3D model data is generated based on the differential data.

[0268] Reference numerals list

[0269] 11 Control Device

[0270] 12-1 to 12-N Camera Devices

[0271] 13 Monitor

[0272] 31 Operating units

[0273] 32 Control Unit

[0274] 33 Reproduction Unit

[0275] 34 Synchronous signal generation unit

[0276] 35 Storage Control Unit

[0277] 36 storage units

[0278] 37 Image Processing Unit

[0279] 41 Reproduction data storage unit

[0280] 42 Imaging data storage unit

[0281] 43 Foreground and background difference processing unit

[0282] 44 3D model data generation unit

[0283] 51 Imaging Unit

[0284] 52 Display units

[0285] 101 Synchronous signal generating device

Claims

1. A control device, comprising: a control unit that performs control to synchronize imaging timing at which the imaging device performs imaging with display timing at which the display device imaged by the imaging device displays the first moving image, and an image processing unit that performs foreground / background difference processing based on a second moving image captured by the imaging device at a first timing and a third moving image captured by the imaging device at a second timing different from the first timing, wherein When imaging the second moving image and the third moving image, the control unit causes the display device to display the same frame image of the first moving image for the same frame number of the second moving image and the third moving image at the first timing and the second timing.

2. The control device according to claim 1, wherein: The second moving image is an actual moving image, and the actual moving image is a moving image including a subject and a background; The third moving image is a background moving image, and the background moving image is a moving image including the background; and The image processing unit performs the foreground / background difference processing using the background moving image and the actual moving image.

3. The control device according to claim 2, wherein: The first moving image is a reference moving image to be referred to by the subject when capturing the actual moving image, and The control unit also performs control to cause the display device to display the reference moving image.

4. The control device according to claim 3, wherein: The reference moving image is a moving image read from a predetermined storage unit to be reproduced.

5. The control device according to claim 4, further comprising a reproduction unit that reproduces the reference moving image stored in the predetermined storage unit, wherein The reproduction unit decodes the reference moving image before a display timing of the display device.

6. The control device according to claim 3, wherein: The reference moving image is a moving image captured by the imaging device when capturing the actual moving image.

7. The control device according to claim 1, wherein: The control unit generates a synchronization signal, causes the imaging device to perform imaging at a rising timing or a falling timing of the synchronization signal, and causes the display device to display the first moving image.

8. The control device according to claim 1, wherein: The control unit performs control to synchronize imaging timings at which the plurality of imaging devices perform imaging with display timings at which the plurality of display devices display the first moving image.

9. The control device according to claim 8, wherein: The control unit causes the plurality of display devices to respectively display different first moving images.

10. The control device according to claim 1, wherein: The control unit also performs control to cause the display device to display information indicating a time code together with the first moving image.

11. The control device according to claim 7, further comprising: A synchronization signal generating unit generates the synchronization signal based on the control of the control unit.

12. The control device according to claim 1, wherein: The image processing unit generates difference data by performing the foreground / background difference processing between the second moving image and the third moving image captured by the imaging device, and further generates 3D model data based on the difference data.

13. An imaging system comprising: multiple imaging devices; at least one display device; a control unit that performs control to synchronize imaging timings at which the plurality of imaging devices perform imaging with display timings at which a display device imaged by at least one of the plurality of imaging devices displays a first moving image; as well as an image processing unit that performs foreground / background difference processing based on a second moving image captured by the imaging device at a first timing and a third moving image captured by the imaging device at a second timing different from the first timing, wherein When imaging the second moving image and the third moving image, the control unit causes the display device to display the same frame image of the first moving image for the same frame number of the second moving image and the third moving image at the first timing and the second timing.

14. A method for generating 3D model data, comprising: synchronizing imaging timings at which a plurality of imaging devices perform imaging with display timings at which a display device imaged by at least one of the plurality of imaging devices displays a first moving image, and performing imaging by the plurality of imaging devices; generating differential data by performing foreground / background differential processing based on a second moving image captured by the imaging device at a first timing and a third moving image captured by the imaging device at a second timing different from the first timing; as well as 3D model data is generated based on the differential data, wherein: When imaging the second moving image and the third moving image, the display device is caused to display the same frame image of the first moving image for the same frame number of the second moving image and the third moving image at the first timing and the second timing. 15 . A non-transitory computer-readable medium having a program embodied thereon, which, when executed by a computer, causes the computer to execute the 3D model data generating method according to claim 14 .

Images