Image processing method, program, and image processing system

Abstract

A video processing method comprises outputting a first superimposed subframe (SF21) having a first pattern image (PP1) superimposed onto a first subframe and a second superimposed subframe (SF22) having a second image pattern (PP2) superimposed onto a second subframe, so as to be displayed on a display surface. The video processing method comprises acquiring, by imaging, the first superimposed subframe (SF21) and the second superimposed subframe (SF22) displayed on the display surface. The video processing method comprises acquiring a third pattern image from the difference between the first superimposed subframe (SF21) and the second superimposed subframe (SF22). The video processing method comprises comparing a feature point of the third pattern image and a reference feature point, thereby detecting a deviation in a display position of the video projected on the display surface. The first subframe and the second subframe are the same image.

Description

【Technical Field】 【0001】 The present disclosure relates to a video processing method, a program, and a video processing system. 【Background Art】 【0002】 Patent Document 1 discloses an image processing method. In this image processing method, a pattern image including a predetermined pattern is superimposed on any one of a plurality of sub-frames corresponding to one frame, and each sub-frame is sequentially projected onto a projection unit. Further, in this image processing method, in synchronization with the control of the projection, an imaging unit images a projection image of the sub-frame on which the pattern image is superimposed and projected by the projection unit. Then, in this image processing method, corresponding points between the projection image and the imaging image are detected based on the pattern image included in the imaging image obtained by imaging by the imaging unit according to the control of the imaging. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 International Publication No. 2017 / 154628 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 The present disclosure provides a video processing method and the like that can easily and accurately detect a shift in the display position of a video without being recognized by a user. 【Means for Solving the Problems】 【0005】 In one aspect of the present disclosure, a video processing method is obtained by temporally dividing a frame contained in video data into three or more subframes. The video processing method outputs a first superimposed subframe, which is obtained by superimposing a first pattern image onto a first subframe based on the plurality of subframes, and a second superimposed subframe, which is obtained by superimposing a second pattern image, which is obtained by inverting the pixel values ​​of the first pattern image onto a second subframe based on the plurality of subframes, so that they are displayed on a display surface. The video processing method acquires the first superimposed subframe and the second superimposed subframe displayed on the display surface by imaging. The video processing method acquires a third pattern image from the difference between the acquired first superimposed subframe and the second superimposed subframe. The video processing method detects a shift in the display position of the video projected onto the display surface by comparing the feature points of the acquired third pattern image with reference feature points. The first subframe and the second subframe are the same image. [Effects of the Invention] 【0006】 This disclosure has the advantage of making it easier to accurately detect misalignments in the display position of video without the user noticing. [Brief explanation of the drawing] 【0007】 [Figure 1] Figure 1 is a schematic diagram of the original image and pattern image projected by the projection device. [Figure 2] Figure 2 is a schematic diagram of the video processing for superimposing a pattern image onto the original image. [Figure 3] Figure 3 is an overview diagram of the pixel shift technology. [Figure 4] Figure 4 is a schematic diagram of the pattern image superposition and extraction processes. [Figure 5] Figure 5 is an explanatory diagram illustrating the problems with the comparative image processing method. [Figure 6] Figure 6 is a schematic diagram showing the overall configuration including the video processing system according to the embodiment. [Figure 7]Figure 7 is a block diagram showing the configuration of a projection device according to an embodiment. [Figure 8] Figure 8 is a flowchart showing an example of the embedding detection process. [Figure 9] Figure 9 is a flowchart showing an example of the process for generating multiple subframes. [Figure 10] Figure 10 shows an example of a pattern image. [Figure 11] Figure 11 is an explanatory diagram illustrating an example of the operation of the image selection unit of a projection device according to an embodiment. [Figure 12] Figure 12 is a schematic diagram of the image projection section of a projection device according to an embodiment. [Figure 13] Figure 13 shows the correlation between the control signal applied to the optical path shift element and the video signal. [Figure 14] Figure 14 is a block diagram showing the configuration of an imaging device according to an embodiment. [Figure 15] Figure 15 is a flowchart showing an example of pattern image detection processing. [Figure 16] Figure 16 is a block diagram showing the configuration of the control device according to the embodiment. [Figure 17] Figure 17 is a flowchart showing an example of the initialization of the misalignment correction process. [Figure 18] Figure 18 shows an example of feature points in the third pattern image. [Figure 19] Figure 19 is a flowchart showing an example of the displacement correction process. [Figure 20] Figure 20 is a flowchart showing an example of the operation of the video processing system according to the embodiment. [Figure 21] Figure 21 is an explanatory diagram illustrating an example of the operation of a projection device according to a first modified embodiment. [Figure 22] Figure 22 is a schematic diagram of the image projection section of a projection device according to a first modified example of the embodiment. [Figure 23] Figure 23 shows the correlation between the control signal applied to the optical path shift element and the video signal in a first modified example of the embodiment. [Figure 24] FIG. 24 is a schematic diagram showing an overall configuration including a video processing system according to a second modification of the embodiment. 【BRIEF DESCRIPTION OF THE DRAWINGS】 【0008】 [1. Knowledge underlying the present disclosure] First, the inventor's focus is described below. 【0009】 Conventionally, in order to correct the distortion of a projection image projected onto a display surface such as a screen by a projection device (projector), that is, the deviation of the display position of the projection image, an imaging device is used to capture the projection image, and geometric correction of the projection image is performed using the captured image. A video processing method is known. The deviation of the display position of the projection image can be caused by, for example, external disturbances such as vibration that cause displacement of the projection device. As such a video processing method, while the user is viewing the video, that is, while the projection device is projecting the video onto the display surface, the inventor of the present application is studying a method of performing geometric correction of the projection image without being recognized by the user. Hereinafter, such a video processing method will be described as the "video processing method of the comparative example". 【0010】 First, the pattern image used in the video processing method of the comparative example will be described. FIG. 1 is a schematic diagram of the original image and the pattern image projected by the projection device. (a) of FIG. 1 is an example of an image included in the video viewed by the user. Here, among the images included in the video, an image on which the pattern image is not superimposed is referred to as the "original image". (b) of FIG. 1 is an example of the pattern image superimposed on the original image. As shown in (b) of FIG. 1, the pattern image includes a predetermined pattern binarized in black and white. 【0011】 In the video processing method of the comparative example, as the pattern images, a first pattern image and a second pattern image are prepared. The first pattern image is an image including a predetermined pattern binarized in black and white. The second pattern image is an image obtained by inverting the luminance value of each pixel of the first pattern image, that is, an image obtained by inverting the black and white of the predetermined pattern included in the first pattern image. 【0012】 Next, we will explain the video processing method of the comparative example. Figure 2 is a schematic diagram of the video processing for superimposing a pattern image onto the original image. In the comparative example's video processing method, first, video data to be projected onto the display surface is acquired at a first frame rate (for example, 60 fps (frames per second)), and then each frame F1 contained in the video data is divided in time to acquire multiple subframes SF1. Here, each frame F1 is divided in time to acquire four subframes SF11, SF12, SF13, and SF14. 【0013】 In the comparative example's video processing method, multiple subframes SF1 are sequentially projected onto the display surface using pixel shift technology, or in other words, wobbling technology, thereby projecting an image onto the display surface at a resolution higher than the resolution supported by the modulation device of the projection device (in this case, 2K resolution) (in this case, 4K resolution). 【0014】 The following describes the pixel shift technique. Figure 3 is an overview diagram of the pixel shift technique. As shown in Figure 3, in the comparative example's video processing method, each frame F1 is divided temporally to obtain multiple subframes SF1 (here, four subframes SF11, SF12, SF13, and SF14). Here, the resolution of frame F1 is 4K, while the resolution of each subframe SF1 is 2K. Subframe SF11 is an image obtained by extracting odd-numbered pixels from the X direction (horizontal direction) and odd-numbered pixels from the Y direction (vertical direction) of frame F1. Subframe SF12 is an image obtained by extracting even-numbered pixels from the X direction and odd-numbered pixels from the Y direction of frame F1. Subframe SF13 is an image obtained by extracting even-numbered pixels from the X direction and even-numbered pixels from the Y direction of frame F1. Subframe SF14 is an image obtained by extracting odd-numbered pixels from the X-direction and even-numbered pixels from the Y-direction of frame F1. Hereafter, subframes SF11, SF12, SF13, and SF14 will also be referred to as subframe "A", subframe "B", subframe "C", and subframe "D", respectively. 【0015】 In the comparative example's video processing method, each subframe SF11, SF12, SF13, and SF14 is projected onto the display surface sequentially, shifting by half a pixel at a second frame rate (for example, 240fps), thereby projecting frame F1' onto the display surface. Frame F1' is an image formed by combining each subframe SF11, SF12, SF13, and SF14, and has a resolution equivalent to that of frame F1 (in this case, 4K resolution). Then, by sequentially projecting each frame F1' corresponding to each frame F1 onto the display surface, the image corresponding to the video data is projected onto the display surface. 【0016】 Next, we will explain the superposition process of pattern images and the extraction process of pattern images in the comparative example's video processing method. In the comparative example's video processing method, as described above, while frame F1' is projected onto the display surface, that is, while multiple subframes SF1 are projected onto the display surface, the first pattern image and the second pattern image are superimposed onto two of the multiple subframes, respectively. Hereafter, the subframe on which the first pattern image is superimposed will also be called the "first superimposed subframe," and the subframe on which the second pattern image is superimposed will also be called the "second superimposed subframe." 【0017】 In the comparative example's image processing method, a pattern image (in this case, the first pattern image) is extracted based on the first and second superimposed subframes captured by the imaging device. 【0018】 Figure 4 is a schematic diagram of the pattern image superposition and extraction process. Figure 4(a) shows the pixel value of the blue signal in one horizontal line of the original image. Figure 4(b) shows the pixel value of the blue signal in the same line of the first pattern image, and Figure 4(c) shows the pixel value of the blue signal in the same line of the second pattern image. In the comparative example video processing method, as described above, the first pattern image and the second pattern image are superimposed on each of two subframes among the multiple subframes. 【0019】 Here, assuming that the two subframes are the same source image, the first superimposed subframe is an image in which the first pattern image is superimposed on the source image, and the second superimposed subframe is an image in which the second pattern image is superimposed on the source image. Figure 4(d) shows the pixel value of the blue signal in the above line of the first superimposed subframe in which the first pattern image is superimposed on the source image, and Figure 4(e) shows the pixel value of the blue signal in the above line of the second superimposed subframe in which the second pattern image is superimposed on the source image. 【0020】 In the comparative example's image processing method, the imaging device captures a first superimposed subframe and a second superimposed subframe, and a difference image is obtained by calculating the difference between the captured image of the first superimposed subframe and the captured image of the second superimposed subframe. Figure 4(f) shows the pixel values ​​of the blue signal in the above line of the difference image. As shown in Figure 4(f), the shape of the pattern in the difference image and the shape of the pattern in the first pattern image are roughly the same. This is because the original image can be removed by calculating the difference between the first superimposed subframe and the second superimposed subframe. Hereafter, this difference image will also be called the "third pattern image". 【0021】 Subsequently, in the comparative example's image processing method, the shift in the display position of the image projected onto the display surface is detected by comparing the feature points of the third pattern image with the reference feature points, and the shift in the display position of the image is corrected according to the detection result. The detection of the shift in the display position of the image and the correction of the shift in the display position of the image will be explained in detail later in [2. Configuration]. 【0022】 Here, in order to accurately detect the shift in the display position of the image, it is necessary to accurately extract the pattern image from the first and second superimposed subframes captured by the imaging device. However, in the video processing method of the comparative example, there is a problem in that it is difficult to accurately extract the pattern image because the subframe on which the first pattern image is superimposed and the subframe on which the second pattern image is superimposed are strictly speaking different images. Specifically, in the video processing method of the comparative example, even if the difference between the first and second superimposed subframes is calculated, it is not possible to remove the high-frequency components from the original image, and the parts that could not be removed are included in the pattern image as noise. 【0023】 Figure 5 is an explanatory diagram of the problems with the comparative example's image processing method. The images shown in Figure 5(a) and (b) are examples of pattern images that contain high-frequency components of the original image as noise. Figure 5(a) is an image created by simulation, and Figure 5(b) is an image captured by the actual device. Ideally, it would be possible to obtain a noise-free pattern image like the one shown in Figure 10(a), which will be described later. However, the comparative example's image processing method obtains a pattern image that contains noise, as shown in Figure 5. 【0024】 Thus, while the video processing method in the comparative example can detect the misalignment of the image display position without the user noticing, it extracts a pattern image containing noise. Therefore, the video processing method in the comparative example detects the misalignment of the image projected onto the display surface based on a comparison between the pattern image containing noise and the reference pattern image, and has the problem that it is difficult to accurately detect the misalignment of the image display position due to the influence of noise. 【0025】 In light of the above, the inventor has created this disclosure. 【0026】 The embodiments will be described below with reference to the drawings. The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit the scope of this disclosure. Furthermore, any components in the following embodiments that are not described in an independent claim will be described as optional components. 【0027】 Please note that each figure is a schematic diagram and not necessarily a strictly accurate representation. Furthermore, in each figure, substantially identical components are denoted by the same reference numerals, and redundant explanations may be omitted or simplified. 【0028】 (Embodiment) [2. Structure] [2-1. Overall Structure] First, the overall configuration including the video processing system 100 according to the embodiment will be described. Figure 6 is a block diagram showing the overall configuration including the video processing system 100 according to the embodiment. The video processing system 100 comprises a projection device 1, an imaging device 2, and a control device 3. The video processing system 100 is a system that processes video data transmitted from the playback device 4. 【0029】 The projection device 1 is a device having a projector function, which projects an image onto the display surface 50 of the screen 5 based on the image data contained in the image signal transmitted from the playback device 4. The projection device 1 is not limited to projecting an image onto the display surface 50 of the screen 5; it may also project an image onto one surface of a structure other than a screen, such as a wall, using that surface as the display surface 50. 【0030】 The imaging device 2 is a device having a camera function that captures the image projected onto the display surface 50. In this embodiment, the imaging device 2 is a separate device from the projection device 1, but it may be built into the projection device 1. 【0031】 The control device 3 is, for example, an information terminal such as a desktop or laptop personal computer, and controls the projection device 1 and the imaging device 2 by communicating with them via a network N1 such as a LAN (Local Area Network). Communication between the projection device 1 and the imaging device 2 and the control device 3 is performed according to a known network protocol such as HTTP (Hypertext Transfer Protocol), FTP (File Transfer Protocol), or TCP (Transmission Control Protocol). 【0032】 In this embodiment, the control device 3 is implemented by installing dedicated software for the video processing system 100 on a general-purpose information terminal. Note that the control device 3 is not limited to a general-purpose information terminal; it may also be an information terminal dedicated to the video processing system 100. Furthermore, the information terminal is not limited to a personal computer; it may be implemented as, for example, a smartphone or tablet device. 【0033】 The playback device 4 is a device that has the function of playing back video recorded on optical media such as DVD (Digital Versatile Disc; registered trademark) or BD (Blu-ray® Disc). The playback device 4 may also be a device that has the function of playing back video recorded on a storage device such as HDD (Hard Disc Drive). 【0034】 [2-2. Projection device] Next, the configuration of the projection device 1 will be described in detail. Figure 7 is a block diagram showing the configuration of the projection device 1 according to an embodiment. As shown in Figure 7, the projection device 1 includes a video input unit 11, a video generation unit 12, a synchronization signal extraction unit 13, a video selection unit 14, a video projection unit 15, a synchronization signal output unit 16, a communication unit 17, a parameter holding unit 18, and a superimposed pattern holding unit 19. The video input unit 11, the video generation unit 12, the synchronization signal extraction unit 13, the video selection unit 14, the video projection unit 15, the synchronization signal output unit 16, and the communication unit 17 may each be implemented by dedicated circuits, or they may be implemented by a processor executing a corresponding computer program stored in memory. 【0035】 The video input unit 11 acquires a video signal input from an external source (in this case, the playback device 4) and converts the acquired video signal into an internal video signal. Here, the resolution and frame rate of the video signal are not particularly limited. In other words, the video input unit 11 receives video signals with various resolutions or frame rates from the playback device 4. In this embodiment, the internal video signal has a resolution of 4K, and the frame rate is a first frame rate (e.g., 60fps), similar to the video processing method in the comparative example. 【0036】 The video generation unit 12 performs various processes on the internal video signal from the video input unit 11. First, the video generation unit 12 performs an embedding determination process to determine whether or not it is possible to embed (superimpose) a pattern image on the internal video signal. In this embodiment, the pattern image is embedded in the blue signal of the internal video signal, which has relatively low brightness sensitivity for humans. In other words, both the pattern images (the first pattern image PP1 and the second pattern image PP2, described later) are superimposed on the blue component of the video signal. Therefore, in this embodiment, the video generation unit 12 performs the embedding determination process on the blue signal of the internal video signal. 【0037】 The embedding detection process will be explained below using Figure 8. Figure 8 is a flowchart of an example of the embedding detection process. The embedding detection process described below is executed for each frame F1. 【0038】 First, the video generation unit 12 counts the number of pixels N in the internal video signal where the signal value (pixel value) of the blue signal is within a predetermined range (S101). Here, the predetermined range is the range between the upper and lower limits of the blue signal value, which is a parameter held in the parameter holding unit 18. When the blue signal value is within the predetermined range, it is possible to embed a pattern image by increasing or decreasing the blue signal value. On the other hand, when the blue signal value is outside the predetermined range, the blue signal becomes saturated when the blue signal value is increased or decreased, and it is not possible to embed a pattern image. 【0039】 Next, the video generation unit 12 compares the counted number of pixels N with the value obtained by multiplying the total number of pixels in frame F1 by the effective ratio (S102). Here, the effective ratio is a parameter held in the parameter holding unit 18 and represents the proportion of pixels in frame F1 to which a pattern image can be embedded. If the number of pixels N is greater than or equal to the value obtained by multiplying the total number of pixels by the effective ratio (S102: Yes), the video generation unit 12 determines that a pattern image can be embedded in frame F1 (S103). On the other hand, if the number of pixels N is less than the value obtained by multiplying the total number of pixels by the effective ratio (S102: No), the video generation unit 12 determines that a pattern image cannot be embedded in frame F1 (S104). 【0040】 The video generation unit 12 executes steps S101 to S104 above if the embedding mode is "enabled". If the embedding mode is "disabled", the video generation unit 12 executes step S104 without executing steps S101 and S102 above. Also, if the embedding mode is "forced", the video generation unit 12 executes step S103 without executing steps S101 and S102 above. Here, the embedding mode is a parameter held in the parameter holding unit 18. 【0041】 Secondly, the video generation unit 12 geometrically corrects the internal video signal according to a LUT (Look Up Table) for geometric correction. This process corrects any misalignment of the display position of the image projected from the projection device 1 onto the display surface 50. Here, the LUT for geometric correction is a set of parameters held in the parameter holding unit 18. 【0042】 Thirdly, the video generation unit 12 executes a generation process to generate multiple subframes SF1 obtained by temporally dividing frame F1. The generation process will be explained below with reference to Figure 9. Figure 9 is a flowchart showing an example of the generation process for multiple subframes SF1. The generation process described below is executed for each frame F1. 【0043】 First, the video generation unit 12 generates subframe "A" (i.e., subframe SF11) by extracting odd-numbered pixels from the X-direction (horizontal direction) and odd-numbered pixels from the Y-direction (vertical direction) of frame F1 (S201). Next, the video generation unit 12 generates subframe "B" (i.e., subframe SF12) by extracting even-numbered pixels from the X-direction and odd-numbered pixels from the Y-direction of frame F1 (S202). Finally, the video generation unit 12 generates subframe "C" (i.e., subframe SF13) by extracting even-numbered pixels from the X-direction and even-numbered pixels from the Y-direction of frame F1 (S203). Furthermore, the video generation unit 12 generates subframe "D" (i.e., subframe SF14) by extracting odd-numbered pixels from the X-direction of frame F1 and even-numbered pixels from the Y-direction of frame F1 (S204). 【0044】 Each of these subframes SF1 is an image composed only of subpixels of the same phase in each pixel of frame F1. For example, if each pixel of frame F1 is composed of four subpixels "A", "B", "C", and "D", then each pixel of subframe "A" is composed only of the subpixel "A" of the corresponding pixel in frame F1. 【0045】 Next, the video generation unit 12 refers to the result of the embedding determination process in frame F1 (S205). If the result of the embedding determination process is that the pattern image cannot be embedded (S205: No), the video generation unit 12 terminates the generation process. On the other hand, if the result of the embedding determination process is that the pattern image can be embedded (S205: Yes), the video generation unit 12 then executes the process to determine the type of pattern image to be embedded in frame F1. 【0046】 Figure 10 shows an example of a pattern image. Figures 10(a) to (c) represent the first pattern image PP1, and Figures 10(d) to (f) represent the second pattern image PP2, respectively. Specifically, Figure 10(a) represents the first pattern image PP11 for the R (red) channel, Figure 10(b) represents the first pattern image PP21 for the G (green) channel, and Figure 10(c) represents the first pattern image PP31 for the B (blue) channel. Furthermore, Figure 10(d) represents the second pattern image PP12 for the R channel, Figure 10(e) represents the second pattern image PP22 for the G channel, and Figure 10(f) represents the second pattern image PP32 for the B channel. 【0047】 In this embodiment, the video generation unit 12 sequentially embeds the first pattern image PP11 and the second pattern image PP12 for the R channel, the first pattern image PP21 and the second pattern image PP22 for the G channel, and the first pattern image PP31 and the second pattern image PP32 for the B channel for each frame F1. 【0048】 Returning to Figure 9, the video generation unit 12 refers to the result of the embedding determination process in the frame before frame F1 (S206). If the pattern image was embeddable in the previous frame (S206: Yes), the video generation unit 12 updates the type of pattern image to be embedded (S207). For example, if the first pattern image PP11 and the second pattern image PP12 for the R channel were embedded in the previous frame, the video generation unit 12 updates the pattern image to be embedded in frame F1 to the first pattern image PP11 for the G channel. 21 and second pattern image PP 22 We have decided on this. 【0049】 On the other hand, if the pattern image could not be embedded in the previous frame (S206: No), the video generation unit 12 initializes the type of pattern image to be embedded (S208). Here, initialization means determining that the pattern images to be embedded in frame F1 are the first pattern image PP11 and the second pattern image PP12 for the R channel. 【0050】 In this way, the video generation unit 12 starts from frame F1 where the pattern image changes from not embeddable to embeddable, and embeds the first pattern image PP11 and the second pattern image PP12 for the R channel into frame F1. Then, as long as the embeddable determination result continues, the video generation unit 12 embeds the first pattern image PP11 and the second pattern image PP12 for the R channel, and the first pattern image PP12 for the G channel. 21 and second pattern image PP 22 , and the first pattern image PP for the B channel 31 and second pattern image PP 32 These are then embedded sequentially for each frame F1. 【0051】 Next, the video generation unit 12 generates subframe "B'" (S209). Here, subframe "B'" is an image in which the first pattern image PP1 is embedded (superimposed) on a composite image obtained by combining subframe "B" and subframe "D". Specifically, the video generation unit 12 generates subframe "B'" by adding the embedded signal value α to the blue signal value of the pixel corresponding to the white color of the first pattern image PP1 for each pixel of the composite image, and by subtracting the embedded signal value α to the blue signal value of the pixel corresponding to the black color of the first pattern image PP1. Here, the embedded signal value α is a parameter held in the parameter holding unit 18. 【0052】 Furthermore, the video generation unit 12 generates a subframe "D'" (S210). Here, subframe "D'" is an image in which the second pattern image PP2 is embedded (superimposed) on a composite image obtained by combining subframe "B" and subframe "D". Specifically, the video generation unit 12 generates subframe "D'" by adding the embedded signal value α to the blue signal value of the pixel corresponding to the white color of the second pattern image PP2 for each pixel of the composite image, and by subtracting the embedded signal value α to the blue signal value of the pixel corresponding to the black color of the second pattern image PP2. 【0053】 The subframe "B'" above corresponds to the first superimposed subframe SF21 (see Figure 11 below), and the subframe "D'" corresponds to the second superimposed subframe SF22 (see Figure 11 below). The composite image obtained by combining subframes "B" and "D" corresponds to both the "first subframe" and the "second subframe". 【0054】 Thus, in this embodiment, the first superimposed subframe SF21 is an image obtained by superimposing the first pattern image PP1 onto the first subframe (here, the composite image) which is based on a plurality of subframes SF1. The second superimposed subframe SF22 is an image obtained by superimposing the second pattern image PP2 onto the second subframe (here, the composite image) which is based on a plurality of subframes SF1. The first subframe and the second subframe are both images obtained by combining two subframes (here, subframes "B" and "D") from the plurality of subframes SF1, and are therefore the same image. On the other hand, subframes SF11 and SF13, on which neither the first pattern image PP1 nor the second pattern image PP2 is superimposed, are both images different from both the first superimposed subframe SF21 and the second superimposed subframe SF22. 【0055】 The synchronization signal extraction unit 13 generates an internal synchronization signal with the same frame rate (in this case, 60fps) as the internal video signal, based on a synchronization signal input from an external source (in this case, the playback device 4) along with the video signal. The internal synchronization signal is supplied to the video generation unit 12, the video selection unit 14, and the video projection unit 15, respectively. The video generation unit 12, the video selection unit 14, and the video projection unit 15 operate frame by frame based on the internal synchronization signal. 【0056】 The video selection unit 14 selects a set of subframes to be projected from the video projection unit 15 onto the display surface 50, according to the result of the embedding determination process in frame F1 in the video generation unit 12. Here, the subframe set consists of multiple subframes SF1 corresponding to frame F1. 【0057】 Figure 11 is an explanatory diagram illustrating an example of the operation of the image selection unit 14 of the projection device 1 according to the embodiment. As shown in Figure 11, if the result of the embedding determination process in frame F1 is that the pattern image cannot be embedded, the image selection unit 14 selects a set of subframes consisting of subframes "A", "B", "C", and "D". Here, subframes "A", "B", "C", and "D" correspond to subframes SF11, SF12, SF13, and SF14, respectively. 【0058】 On the other hand, as shown in Figure 11, if the result of the embedding determination process in frame F1 is that the pattern image can be embedded, the video selection unit 14 selects a set of subframes consisting of subframes "A", "B'", "C", and "D'". Here, subframes "A", "B'", "C", and "D'" correspond to subframe SF11, first superimposed subframe SF21, subframe SF13, and second superimposed subframe SF22, respectively. 【0059】 As described above, in this embodiment, the video selection unit 14 selects a set of subframes to be output to the display surface 50 according to the result of the embedding determination process in the video generation unit 12. Furthermore, as already mentioned, in the embedding determination process, for each frame F1, it is determined whether or not it is possible to embed a pattern image by referring to the signal value of the blue signal among the internal video signals. In other words, the video processing system 100 according to this embodiment decides whether or not to output the first superimposed subframe SF21 and the second superimposed subframe SF22 to the display surface 50 based on the pixel value of the video signal in frame F1 (in this case, the signal value of the blue signal). 【0060】 The video projection unit 15 projects an image onto the display surface 50 according to the video signal of the subframe set selected by the video selection unit 14. The specific configuration and operation of the video projection unit 15 will be described below with reference to Figures 12 and 13. Figure 12 is a schematic diagram of the video projection unit 15 of the projection device 1 according to the embodiment. Figure 13 is a diagram showing the correlation between the control signal given to the optical path shift element 153 and the video signal. 【0061】 As shown in Figure 12, the image projection unit 15 includes a light source 151, a modulation device 152, an optical path shift element 153, and a projection lens 154. 【0062】 The light source 151 includes, for example, an ultra-high pressure mercury lamp or a metal halide lamp, and outputs parallel light to the modulation device 152. 【0063】 The modulation device 152 modulates the light output from the light source 151 according to the input video signal and outputs the modulated light to the optical path shift element 153. 【0064】 The optical path shift element 153 is made of, for example, a translucent parallel plate glass and tilts according to the signal voltage of the control signal. The optical path of light incident on the optical path shift element 153 is shifted according to the tilt of the optical path shift element 153. In this embodiment, the control signal includes a horizontal control signal and a vertical control signal. Therefore, the optical path shift element 153 can tilt in either the horizontal or vertical direction according to the signal voltage of the control signal. 【0065】 The projection lens 154 focuses the light output from the optical path shift element 153 and outputs it to the display surface 50, forming an image on the display surface 50 corresponding to the light output from the optical path shift element 153. 【0066】 In this embodiment, similar to the video processing method of the comparative example, image shifting technology is used to sequentially project each subframe SF1 included in the subframe set selected by the video selection unit 14 onto the display surface 50, shifting it by half a pixel at a second frame rate (here, 240 fps). As shown in Figure 13, both the horizontal control signal and the vertical control signal are rectangular wave signals that alternate between high and low levels in the first period Td1 (here, 1 / 120 seconds). The horizontal control signal and the vertical control signal are out of phase with respect to each other by 1 / 4 of the first period Td1. Therefore, the combination of the signal voltage of the horizontal control signal and the signal voltage of the vertical control signal changes in the second period Td2 (here, 1 / 240 seconds). 【0067】 For example, the image projection unit 15 projects light corresponding to subframe "A" onto the display surface 50 at the timing when the horizontal control signal and the vertical control signal are at a high level. As a result, subframe "A" is projected onto the display surface 50. 【0068】 Furthermore, the image projection unit 15 projects light corresponding to subframe "B" or subframe "B'" onto the display surface 50 at the timing when the horizontal control signal is at a low level and the vertical control signal is at a high level. As a result, subframe "B" or subframe "B'" is projected onto the display surface 50 at a position that is shifted by half a pixel horizontally from the display position of subframe "A". 【0069】 Furthermore, the image projection unit 15 projects light corresponding to subframe "C" onto the display surface 50 at the timing when the horizontal control signal and the vertical control signal are at a low level. As a result, subframe "C" is projected onto the display surface 50 at a position that is shifted by half a pixel horizontally and half a pixel vertically compared to the display position of subframe "A". 【0070】 Furthermore, the image projection unit 15 projects light corresponding to subframe "D" or subframe "D'" onto the display surface 50 at the timing when the horizontal control signal is at a high level and the vertical control signal is at a low level. As a result, subframe "D" or subframe "D'" is projected onto the display surface 50 at a position that is half a pixel vertically shifted from the display position of subframe "A". 【0071】 In this way, the video processing system 100 according to the embodiment uses image shifting technology to sequentially project a plurality of subframes SF1 (here, subframe "A", subframe "B" (or "B'"), subframe "C", subframe "D" (or "D'")) onto the display surface 50. As a result, the video processing system 100 according to the embodiment projects an image onto the display surface 50 at a resolution higher than the resolution that the modulation device 152 of the projection device 1 can handle (here, a resolution of 2K) (here, a resolution of 4K). 【0072】 The synchronization signal output unit 16 outputs a synchronization signal to the imaging device 2. The synchronization signal is a pulse signal that becomes high level at the timing when the first superimposed subframe SF21 and the second superimposed subframe SF22 are projected onto the display surface 50. If the subframe set selected by the video selection unit 14 does not include the first superimposed subframe SF21 and the second superimposed subframe SF22, the synchronization signal output unit 16 does not output a synchronization signal to the imaging device 2. 【0073】 The communication unit 17 is a communication interface for communicating with the control device 3 via the network N1. The communication unit 17 receives parameter setting commands transmitted from the control device 3 and changes various parameters held in the parameter holding unit 18 according to the content of the received parameter setting commands. The communication between the communication unit 17 and the control device 3 may be wired communication or wireless communication. 【0074】 The parameter holding unit 18 is a semiconductor memory or the like, and holds various parameters that the projection device 1 references when it operates. In this embodiment, the parameter holding unit 18 holds the upper and lower limits of the blue signal value, the effective ratio, the embedded signal value α, and the embedded mode, which are parameters referenced in the embedding determination process described above. The parameter holding unit 18 also holds the LUT for geometric correction described above. Note that these parameters are just examples, and the parameter holding unit 18 may hold other parameters as well. 【0075】 The superimposed pattern holding unit 19 is a semiconductor memory or the like, and holds bitmap data of pattern images (first pattern image PP1 and second pattern image PP2) to be superimposed on the subframe SF1. Note that the parameter holding unit 18 and the superimposed pattern holding unit 19 may be implemented by the same semiconductor memory. 【0076】 [2-3. Imaging device] Next, the configuration of the imaging device 2 will be described in detail. Figure 14 is a block diagram showing the configuration of the imaging device 2 according to an embodiment. As shown in Figure 14, the imaging device 2 includes a communication unit 21, a screen generation unit 22, a synchronization signal input unit 23, an imaging unit 24, a pattern detection unit 25, a parameter holding unit 26, and a superimposed pattern holding unit 27. The communication unit 21, the screen generation unit 22, the synchronization signal input unit 23, the imaging unit 24, and the pattern detection unit 25 may each be implemented by dedicated circuits, or they may be implemented by a processor executing a corresponding computer program stored in memory. 【0077】 The communication unit 21 is a communication interface for communicating with the control device 3 via the network N1. The communication unit 21 receives commands transmitted from the control device 3 and relays the received commands to the screen generation unit 22. The communication unit 21 also transmits the processing results executed by the screen generation unit 22 to the control device 3. The communication between the communication unit 21 and the control device 3 may be wired or wireless. 【0078】 The screen generation unit 22 generates a screen to be displayed on the display of the control device 3 by the screen display unit 32 (described later) of the control device 3. In this embodiment, the screen generation unit 22 generates an HTML page in response to a command from the control device 3. For example, the screen generation unit 22 generates an HTML page in response to a command from the control device 3 that includes the current various parameters of the imaging device 2 and an icon that accepts changes to the various parameters. Alternatively, for example, the screen generation unit 22 executes a process to change the various parameters of the imaging device 2, or a process to start or end imaging by the imaging unit 24, in response to a command from the control device 3, and generates an HTML page that includes the processing results. 【0079】 The synchronization signal input unit 23 receives the synchronization signal transmitted from the projection device 1 and provides the received synchronization signal to the imaging unit 24. 【0080】 The imaging unit 24 captures the image projected onto the display surface 50. In this embodiment, the imaging unit 24 starts exposure at a timing corresponding to the trigger mode. Here, the trigger mode is a parameter held by the parameter holding unit 26. When the trigger mode is "synchronization signal", the imaging unit 24 starts exposure at the timing when the pulse of the synchronization signal from the projection device rises. In other words, in this case, the imaging unit 24 captures only the first superimposed subframe SF21 and the second superimposed subframe SF22 of the image projected onto the display surface 50. When the trigger mode is "program", the imaging unit 24 starts exposure after receiving a command to start imaging from the control device 3. 【0081】 The time from when the imaging unit 24 starts exposure until it finishes is determined by the exposure time (in milliseconds) held by the parameter holding unit 26. Also, if the trigger delay amount (in microseconds) held by the parameter holding unit 26 is not zero, the imaging unit 24 starts exposure after the synchronization signal pulse rises, with a delay of the trigger delay amount. 【0082】 The pattern detection unit 25 performs a detection process to detect a pattern image from the first superimposed subframe SF21 and the second superimposed subframe SF22 captured by the imaging unit 24. The detection process will be explained below with reference to Figure 15. Figure 15 is a flowchart showing an example of the pattern image detection process. The detection process described below is performed each time the first superimposed subframe SF21 and the second superimposed subframe SF22 are captured by the imaging unit 24. 【0083】 First, the pattern detection unit 25 obtains a difference image by calculating the difference between the first superimposed subframe SF21 and the second superimposed subframe SF22 captured by the imaging unit 24 (S301). Note that the display position of the first superimposed subframe SF21 and the display position of the second superimposed subframe SF22 on the display surface 50 are shifted by the amount of shift by the optical path shift element 153 because the image projection unit 15 of the projection device 1 uses pixel shift technology. Therefore, the pattern detection unit 25 shifts either the first superimposed subframe SF21 or the second superimposed subframe SF22 by the amount of the above shift and then calculates the difference. 【0084】 Here, the difference image acquired in step S301 is one of the pattern image for the R channel, the pattern image for the G channel, or the pattern image for the B channel. For example, if the projection device 1 has a first pattern image PP11 and a second pattern image PP12 for the R channel embedded in frame F1, the pattern detection unit 25 will acquire the pattern image for the R channel when the video corresponding to frame F1 is projected onto the display surface 50. 【0085】 Next, the pattern detection unit 25 averages the multiple difference images (S302). Here, the pattern detection unit 25 sequentially acquires a difference image corresponding to the pattern image for the R channel, a difference image corresponding to the pattern image for the G channel, and a difference image corresponding to the pattern image for the B channel for each frame F1. Therefore, as long as a pattern image is embedded in each frame F1 in the projection device 1, the pattern detection unit 25 can acquire a difference image corresponding to the pattern image for the same channel every three frames. Then, when the pattern detection unit 25 has acquired a predetermined number of difference images (for example, 10) for each of the R channel, G channel, and B channel, it averages these multiple difference images. This makes it possible to reduce the noise contained in the averaged difference image. 【0086】 Next, the pattern detection unit 25 binarizes the averaged difference image (S303). Here, since the first superimposed subframe SF21 and the second superimposed subframe SF22 captured by the imaging unit 24 are both color images, the averaged difference image is also a color image. Therefore, the pattern detection unit 25 obtains a black and white binarized difference image by binarizing the averaged difference image. 【0087】 Next, the pattern detection unit 25 determines the type of pattern image by pattern matching the grayscale binarized difference image with the pattern image templates for the R channel, the G channel, and the B channel held by the superimposed pattern holding unit 27 (S304). For example, if the grayscale binarized difference image and the pattern image template for the G channel are roughly the same, the pattern detection unit 25 determines that the difference image is a pattern image for the G channel. 【0088】 Then, the pattern detection unit 25 writes the difference image, whose pattern image type has been determined, to memory as the third pattern image PP3 (S305). By repeating the above steps S301 to S305, the imaging device 2 acquires the third pattern image PP3 for the R channel, the third pattern image PP3 for the G channel, and the third pattern image PP3 for the B channel. 【0089】 The parameter holding unit 26 is a semiconductor memory or the like, and holds various parameters that the imaging device 2 references when it operates. In this embodiment, the parameter holding unit 26 holds the trigger mode, exposure time, and trigger delay amount, as already described. Note that these parameters are just examples, and the parameter holding unit 26 may hold other parameters as well. 【0090】 The superimposed pattern holding unit 27 is a semiconductor memory or the like, and holds bitmap data of the pattern image templates for the R channel, the G channel, and the B channel used in the detection process described above. Note that the parameter holding unit 26 and the superimposed pattern holding unit 27 may be implemented by the same semiconductor memory. 【0091】 [2-4. Control Device] Next, the configuration of the control device 3 will be described in detail. Figure 16 is a block diagram showing the configuration of the control device 3 according to an embodiment. As shown in Figure 16, the control device 3 includes an input unit 31, a screen display unit 32, a communication unit 33, a misalignment correction unit 34, and a data storage unit 35. The input unit 31, the screen display unit 32, the communication unit 33, and the misalignment correction unit 34 may each be implemented by dedicated circuits, or they may be implemented by a processor executing a corresponding computer program stored in memory. 【0092】 The input unit 31 accepts input from the user using, for example, a keyboard or a pointing device such as a mouse. The input unit 31 provides control commands to the projection device 1 or imaging device 2 in response to the user input. The control commands include, for example, instructions to change various parameters of the imaging device 2, instructions to transmit various parameters of the imaging device 2, instructions to transmit the third pattern image PP3 from the imaging device 2, instructions to initialize the misalignment correction process by the misalignment correction unit 34 (described later), or instructions to start or end the misalignment correction process by the misalignment correction unit 34. 【0093】 The screen display unit 32 displays a UI (User Interface) screen for operating the control device 3 on the display attached to the control device 3. For example, the screen display unit 32 displays an HTML page or the like generated by the screen generation unit 22 of the imaging device 2 on the display. 【0094】 The communication unit 33 is a communication interface for communicating with the projection device 1 and the imaging device 2, respectively, via the network N1. The communication unit 33 transmits control commands to either the projection device 1 or the imaging device 2. The communication unit 33 also transmits corrected LUT data for geometric correction, corrected by the shift correction process described later, to the projection device 1. The communication between the communication unit 33 and the projection device, and the communication between the communication unit 33 and the imaging device 2, may be wired or wireless. 【0095】 The misalignment correction unit 34 has a function to initialize the misalignment correction process. The initialization of the misalignment correction process will be explained below with reference to Figure 17. Figure 17 is a flowchart showing an example of the initialization of the misalignment correction process. The initialization of the misalignment correction process only needs to be performed once in response to user input received by the input unit 31 before executing the misalignment correction process, for example, when starting to use the video processing system 100. 【0096】 First, the misalignment correction unit 34 acquires LUT data for geometric correction from the projection device 1 (S401). Next, the misalignment correction unit 34 acquires the third pattern image PP3 for the R channel, the third pattern image PP3 for the G channel, and the third pattern image PP3 for the B channel from the imaging device 2, and detects the feature points SP1 of the third pattern image PP3 from these images (S402). 【0097】 The method for detecting feature points SP1 in the third pattern image PP3 will be explained below using Figure 18. Figure 18 shows an example of feature points SP1 in the third pattern image PP3. Figure 18 shows the third pattern image PP3 obtained by combining the third pattern image PP3 for the R channel, the third pattern image PP3 for the G channel, and the third pattern image PP3 for the B channel. Here, the white pixels in the third pattern image PP3 for the R channel are changed to red, the white pixels in the third pattern image PP3 for the G channel are changed to green, and the white pixels in the third pattern image PP3 for the B channel are changed to blue, and the third pattern image PP3 for each channel is combined. 【0098】 In Figure 18, each pixel is color-coded according to the type of hatching. Here, feature point SP1 is the intersection of four regions where the colors of the upper region, lower region, right region, and left region are all different from each other. In particular, in the synthesized third pattern image PP3, there is only one point where the colors of the upper region, lower region, right region, and left region are in a specific combination. In this embodiment, the misalignment correction unit 34 detects the intersection of the four regions where the color combinations are in the specific combination described above as feature point SP1. 【0099】 Returning to Figure 17, the displacement correction unit 34 stores data linking each point of the geometric correction LUT with the detected feature points SP1 of the third pattern image PP3 as initial data in the data storage unit 35 (S403). 【0100】 Furthermore, the misalignment correction unit 34 has a function to perform misalignment correction processing. The misalignment correction processing will be explained below with reference to Figure 19. Figure 19 is a flowchart of an example of misalignment correction processing. In the following, the misalignment correction processing will be performed in response to user input received by the input unit 31 after the initialization of the misalignment correction processing has been performed. Note that the misalignment correction processing may be performed periodically regardless of user input. 【0101】 If the misalignment correction unit 34 has not received a termination instruction from the user (S501: No), it repeats the series of processes from steps S502 to S507 shown below. On the other hand, if the misalignment correction unit 34 receives a termination instruction from the user (S501: Yes), it terminates the misalignment correction process. 【0102】 First, the misalignment correction unit 34 waits until the pattern image (third pattern image PP3 for each channel) acquired from the imaging device 2 is updated (S502: No). Then, when the pattern image acquired from the imaging device 2 is updated (S502: Yes), the misalignment correction unit 34 detects feature points SP1 based on the acquired third pattern image PP3 for each channel (S503). The method for detecting feature points SP1 has already been described, so the explanation is omitted here. 【0103】 Next, the misalignment correction unit 34 compares the detected feature point SP1 with the feature point SP1 included in the initial data (S504). Here, the misalignment correction unit 34 compares the XY plane coordinates of the detected feature point SP1 with the XY plane coordinates of the feature point SP1 included in the initial data. 【0104】 If the comparison shows no shift in the position of feature point SP1 (S505: No), the shift correction unit 34 does not update the LUT for geometric correction and the initial data. On the other hand, if there is a shift in the position of feature point SP1 (S505: Yes), the shift correction unit 34 generates a LUT for geometric correction that makes the shift zero and updates the LUT for geometric correction (S506). The shift correction unit 34 also updates the initial data using the updated LUT for geometric correction (S507). Specifically, the shift correction unit 34 updates the detected feature point SP1 as the feature point SP1 included in the initial data. 【0105】 At this time, the misalignment correction unit 34 transmits the updated (corrected) geometric correction LUT data to the projection device 1 via the communication unit 33 and the network N1. The projection device 1 then geometrically corrects the internal video signal according to the acquired corrected geometric correction LUT. This makes it possible to correct the misalignment of the display position of the image on the display surface 50. 【0106】 The data storage unit 35 is a semiconductor memory or the like, and stores initial data including geometric correction LUT data acquired from the imaging device 2, and third pattern images PP3 for each channel acquired from the imaging device 2. 【0107】 [3. Operation] The overall operation of the video processing system 100 according to this embodiment, that is, the video processing method according to this embodiment, will be explained below using Figure 20. Figure 20 is a flowchart showing an example of the operation of the video processing system 100 according to this embodiment. 【0108】 First, the video processing system 100 acquires three or more subframes SF1, which are obtained by temporally dividing the frame F1 contained in the video data (S1). In this embodiment, the entity that executes step S1 is the video generation unit 12 of the projection device 1. 【0109】 Next, the video processing system 100 outputs the first superimposed subframe SF21 and the second superimposed subframe SF22 to be displayed on the display surface 50 (S2). The first superimposed subframe SF21 is an image in which the first pattern image PP1 is superimposed on the first subframe which is based on multiple subframes SF1. The second superimposed subframe SF 22 This is an image in which a second pattern image PP2 is superimposed on a second subframe based on multiple subframes SF1, with the pixel values ​​of the first pattern image PP1 inverted. In this embodiment, the main entities executing step S2 are the image generation unit 12, the image selection unit 14, and the image projection unit 15 of the projection device 1. 【0110】 Next, the video processing system 100 acquires the first superimposed subframe SF21 and the second superimposed subframe SF22 displayed on the display surface 50 by imaging (step S3). In this embodiment, the main entities performing step S3 are the imaging unit 24 and the pattern detection unit 25 of the imaging device 2. 【0111】 Next, the video processing system 100 acquires a third pattern image PP3 from the difference between the acquired first superimposed subframe SF21 and the second superimposed subframe SF22 (step S4). In this embodiment, the main entity executing step S4 is the pattern detection unit 25 of the imaging device 2. 【0112】 Then, the video processing system 100 detects the shift in the display position of the image projected onto the display surface 50 by comparing the acquired feature points SP1 of the third pattern image PP3 with reference feature points (S5). Here, the reference feature points are the feature points SP1 of the third pattern image PP3 included in the initial data described above. In this embodiment, the main entity executing step S5 is the shift correction unit 34 of the control device 3. 【0113】 In this embodiment, the video processing system 100 performs a process to update the LUT for geometric correction in order to correct the detected shift in the display position, but this process does not need to be performed. 【0114】 [4. Advantages, etc.] The advantages of the image processing system 100 (image processing method) according to the embodiment will be described below. As described above, in the image processing system 100 according to the embodiment, a pattern image (third pattern image PP3) is extracted based on the first superimposed subframe SF21 and the second superimposed subframe SF22 captured by the imaging device 2, similar to the image processing method of the comparative example. In the image processing system 100 according to the embodiment, the first subframe, which is the image on which the first pattern image PP1 is superimposed when the first superimposed subframe SF21 is generated, and the second subframe, which is the image on which the second pattern image PP2 is superimposed when the second superimposed subframe SF22 is generated, are the same image. 【0115】 Therefore, in the video processing system 100 according to this embodiment, when calculating the difference between the first superimposed subframe SF21 and the second superimposed subframe SF22 to obtain the third pattern image PP3, it is easier to remove high-frequency components from the original image, and noise is less likely to be included in the third pattern image PP3. Consequently, the video processing system 100 according to this embodiment can extract the third pattern image PP3 with high accuracy, which has the advantage of being able to accurately detect shifts in the display position of the video without the user noticing. 【0116】 [5. Other Embodiments] Although embodiments have been described above, this disclosure is not limited to the embodiments described above. 【0117】 [5-1. First variation] For example, in the above embodiment, the first subframe and the second subframe are both images obtained by combining two subframes from a plurality of subframes SF1, but the embodiment is not limited to this. For example, the first subframe and the second subframe may both be one subframe from a plurality of subframes SF1. 【0118】 The following description of this first modified example will be made using Figures 21 to 23. Figure 21 is an explanatory diagram of an example of operation of the projection device 1 according to the first modified example of the embodiment. Figure 22 is a schematic diagram of the image projection unit 15 of the projection device 1 according to the first modified example of the embodiment. Figure 23 is a diagram showing the correlation between the control signal given to the optical path shift element 153 and the image signal in the first modified example of the embodiment. In the following description, points common to the image processing system 100 according to the embodiment will be omitted. 【0119】 As shown in Figure 21, in the first modified example, the video selection unit 14 selects a set of subframes consisting of subframes "A", "A", "C'", and "C''" if the result of the embedding determination process in frame F1 is that the pattern image can be embedded. Here, subframes "A", "C'", and "C''" correspond to subframe SF11, first superimposed subframe SF21, and second superimposed subframe SF22, respectively. 【0120】 In other words, in the first modified example, the image generation unit 12 generates subframes "C'" and "C''" instead of subframes "B'" and "D'". Here, subframe "C'" is an image in which the first pattern image PP1 is embedded (superimposed) on subframe "C". Also, subframe "C''" is an image in which the second pattern image PP2 is embedded on subframe "C". In other words, in the first modified example, both the first subframe and the second subframe are one of multiple subframes SF1 (in this case, subframe "C"). 【0121】 Furthermore, in the first modified example, if the result of the embedding determination process in frame F1 indicates that the pattern image can be embedded, the image projection unit 15, as shown in Figure 22, uses a different pixel shift technique than in the embodiment to sequentially project each subframe SF1 included in the subframe set selected by the image selection unit 14 onto the display surface 50 while shifting it at a third frame rate (here, 120 fps). 【0122】 Specifically, as shown in Figure 23, the image projection unit 15 projects light corresponding to subframe "A" onto the display surface 50 continuously when the horizontal control signal is at a high level and the vertical control signal is at a high level. As a result, subframe "A" is continuously projected onto the display surface 50. 【0123】 Furthermore, when the horizontal control signal and the vertical control signal are at a low level, the image projection unit 15 first projects light corresponding to subframe "C'" onto the display surface 50, and then projects light corresponding to subframe "C''" onto the display surface 50. As a result, subframes "C'" and "C''" are projected onto the display surface 50 at positions that are shifted by half a pixel horizontally and vertically compared to the display position of subframe "A". 【0124】 In the first modified example, as in the embodiment, noise is less likely to be included in the third pattern image PP3, and the third pattern image PP3 can be extracted with high accuracy. This has the advantage of making it easier to accurately detect shifts in the display position of the video without the user noticing. 【0125】 [5-2. Second variation] For example, in the above embodiment, there is one projection device 1, but it is not limited to this. For example, there may be multiple projection devices 1. 【0126】 The second modified example will be described below with reference to Figure 24. Figure 24 is a schematic diagram showing the overall configuration including the video processing system 100 according to the second modified example of the embodiment. As shown in Figure 24, in the second modified example, images are projected onto the display surface 50 from multiple projection devices 1 (here, two projection devices 1A and 1B), and a composite image is projected onto the display surface 50. In such a case, the video processing system 100 may have the control device 3 control each projection device 1A and 1B to alternately execute the process of outputting the first superimposed subframe SF21 and the second superimposed subframe SF22 to the display surface 50. 【0127】 In the second modified example, the first superimposed subframe SF21 and the second superimposed subframe SF22 projected from each projection device 1 do not overlap on the display surface 50, which has the advantage that the third pattern image PP3 is less likely to contain noise. 【0128】 [5-3. Other variations] For example, in the above embodiment, the video processing system 100 is implemented by multiple devices, but it is not limited to this. For example, the video processing system 100 may be implemented by a single device. 【0129】 Furthermore, in the above embodiment, the processing performed by a specific processing unit may be performed by another processing unit. Also, the order of multiple processing units may be changed, or multiple processing units may be executed in parallel. 【0130】 Furthermore, in the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. 【0131】 Furthermore, each component may be implemented by hardware. Each component may also be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or they may be separate circuits. Also, each of these circuits may be a general-purpose circuit or a dedicated circuit. 【0132】 Furthermore, the general or specific embodiments of this disclosure may be implemented as a system, apparatus, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM. They may also be implemented in any combination of systems, apparatus, methods, integrated circuits, computer programs, and recording media. 【0133】 Furthermore, this disclosure may be implemented as a video processing method executed by a computer, such as the video processing system of the above embodiment. This disclosure may be implemented as a program (computer program product) for causing a computer to execute such a video processing method, or as a computer-readable non-temporary recording medium on which such a program is recorded. 【0134】 Furthermore, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art could conceive, or forms realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of this disclosure. 【0135】 (summary) As described above, in the first embodiment of the video processing method, a plurality of subframes SF1, which are three or more in number, are obtained by temporally dividing the frame F1 contained in the video data. Furthermore, in this video processing method, a first superimposed subframe SF21 is obtained by superimposing a first pattern image PP1 onto a first subframe based on the plurality of subframes SF1, and a second superimposed subframe SF22 is obtained by superimposing a second pattern image PP2, which is obtained by inverting the pixel values ​​of the first pattern image PP1 onto a second subframe based on the plurality of subframes SF1, and these are output to be displayed on the display surface 50. Furthermore, in this video processing method, the first superimposed subframe SF21 and the second superimposed subframe SF22 displayed on the display surface 50 are obtained by imaging. Furthermore, in this video processing method, a third pattern image PP3 is obtained from the difference between the obtained first superimposed subframe SF21 and the second superimposed subframe SF22. Furthermore, this image processing method detects the shift in the display position of the image projected onto the display surface 50 by comparing the feature points SP1 of the acquired third pattern image PP3 with reference feature points. The first subframe and the second subframe are the same image. 【0136】 This type of video processing method has the advantage of being less prone to noise in the third pattern image PP3 and being able to extract the third pattern image PP3 with high accuracy, making it easier to accurately detect shifts in the display position of the video without the user noticing. 【0137】 Furthermore, for example, in the image processing method according to the second embodiment, in the first embodiment, each of the multiple subframes SF1 is an image composed only of subpixels of the same phase in each pixel of frame F1. 【0138】 This type of image processing method has the advantage of making it easier to create identical images for the first and second subframes. 【0139】 Furthermore, for example, in the image processing method according to the third embodiment, in the second embodiment, both the first subframe and the second subframe are images obtained by combining two subframes SF1 from a plurality of subframes SF1. 【0140】 This image processing method has the advantage of making the first and second subframes the same image while maintaining the quality of the image projected onto the display surface 50. 【0141】 Furthermore, for example, in the video processing method according to the fourth embodiment, in the second embodiment, both the first subframe and the second subframe are one of a plurality of subframes SF1. 【0142】 This type of image processing method has the advantage of making it easier to create identical images for the first and second subframes. 【0143】 Furthermore, for example, in the video processing method according to the fifth embodiment, in any one of the first to fourth embodiments, the first pattern image PP1 and the second pattern image PP2 are both superimposed on the blue component video signal. 【0144】 This type of image processing has the advantage of superimposing a pattern image onto the blue light signal, which humans are relatively less sensitive to in terms of brightness, making the pattern image less likely to be perceived by the user. 【0145】 Furthermore, for example, in the video processing method according to the sixth embodiment, in any one of the first to fifth embodiments, it is determined whether or not to output the first superimposed subframe SF21 and the second superimposed subframe SF22 to the display surface 50 based on the pixel values ​​of the video signal in frame F1. 【0146】 This type of video processing method has the advantage that the video signal is less likely to saturate when a pattern image is superimposed on it, making it easier to superimpose the pattern image without distorting it. 【0147】 Furthermore, in the image processing method according to the seventh embodiment, for example, in any one of the first to sixth embodiments, there are multiple projection devices 1 that project an image onto the display surface 50. In this image processing method, when multiple projection devices 1 project images onto the display surface 50 to create a composite image, the process of outputting the first superimposed subframe SF21 and the second superimposed subframe SF22 to the display surface 50 is performed alternately by the multiple projection devices 1. 【0148】 This image processing method has the advantage that it is easier to obtain a noise-free third pattern image PP3 because multiple projection devices 1 do not simultaneously perform the process of outputting the first superimposed subframe SF21 and the second superimposed subframe SF22 to the display surface 50. 【0149】 Furthermore, for example, the program relating to the eighth embodiment causes one or more processors to execute the video processing method of any one of the first to seventh embodiments. 【0150】 Such programs have the advantage of being less prone to noise in the third pattern image PP3 and being able to extract the third pattern image PP3 with high accuracy, making it easier to accurately detect shifts in the display position of the video without the user noticing. 【0151】 Furthermore, for example, the video processing system 100 according to the ninth embodiment includes a first acquisition unit (video generation unit 12 of the projection device 1), an output unit (video generation unit 12, video selection unit 14, and video projection unit 15 of the projection device 1), a second acquisition unit (imaging unit 24 and pattern detection unit 25 of the imaging device 2), a third acquisition unit (pattern detection unit 25 of the imaging device 2), and a detection unit (shift correction unit 34 of the control device 3). The first acquisition unit acquires three or more subframes SF1 obtained by temporally dividing a frame F1 contained in the video data. The output unit outputs a first superimposed subframe SF21 obtained by superimposing a first pattern image PP1 onto a first subframe based on the multiple subframes SF1, and a second superimposed subframe SF22 obtained by superimposing a second pattern image PP2 obtained by inverting the pixel values ​​of the first pattern image PP1 onto a second subframe based on the multiple subframes SF1, so as to display on the display surface 50. The second acquisition unit acquires the first superimposed subframe SF21 and the second superimposed subframe SF22 displayed on the display surface 50 by imaging. The third acquisition unit acquires the third pattern image PP3 from the difference between the acquired first superimposed subframe SF21 and the second superimposed subframe SF22. The detection unit detects the shift in the display position of the image projected onto the display surface 50 by comparing the feature points SP1 of the acquired third pattern image PP3 with reference feature points. The first subframe and the second subframe are the same image. 【0152】 Such an image processing system 100 has the advantage that noise is less likely to be included in the third pattern image PP3, and the third pattern image PP3 can be extracted with high accuracy, making it easy to accurately detect shifts in the display position of the image without the user noticing. [Explanation of symbols] 【0153】 100 Video Processing Systems 1, 1A, 1B Projection devices 11. Video Input Section 12. Video Generation Unit 13 Synchronization signal extraction unit 14. Video Selection Section 15. Video Projection Unit 151 Light source 152 Modulation Devices 153 Optical path shift element 154 projection lens 16 Synchronization signal output section 17 Communications Department 18 Parameter holding unit 19. Superimposed pattern holding section 2. Imaging device 21 Communications Department 22 Screen generation section 23 Synchronization signal input section 24 Imaging Department 25 Pattern detection unit 26 Parameter holding unit 27 Superimposed pattern holding section 3. Control device 31 Input section 32 Screen display section 33 Communications Department 34. Misalignment correction unit 35 Data Storage Unit 4 Playback device 5 screens 50 Display surface F1, F1' frame N1 Network PP1, PP11, PP21, PP31 First Pattern Images PP2, PP12, PP22, PP32 Second Pattern Images PP3 Third Pattern Image SF1, SF11, SF12, SF13, SF14 subframes SF21 1st Superimposed Subframe SF22 Second Overlay Subframe SP1 Feature Points Td1 1st period Td2 2nd period

Claims

[Claim 1] By dividing the frame contained in the video data into three or more subframes in time, A first superimposed subframe, obtained by superimposing a first pattern image onto a first subframe based on the plurality of subframes, and a second superimposed subframe, obtained by superimposing a second pattern image obtained by inverting the pixel values ​​of the first pattern image onto a second subframe based on the plurality of subframes, are output to be displayed on the display surface. The first superimposed subframe and the second superimposed subframe displayed on the display surface are acquired by imaging. A third pattern image is obtained from the difference between the acquired first superimposed subframe and the acquired second superimposed subframe. By comparing the feature points of the acquired third pattern image with the reference feature points, a shift in the display position of the image projected onto the display surface is detected. Each of the aforementioned subframes is an image composed of subpixels of the same phase in each pixel of the frame. Image processing methods. [Claim 2] The first subframe and the second subframe are the same image. The image processing method according to claim 1. [Claim 3] The first subframe and the second subframe are both images obtained by combining two subframes from the plurality of subframes. The image processing method according to claim 2. [Claim 4] The first subframe and the second subframe are both subframes among the plurality of subframes. The image processing method according to claim 2. [Claim 5] Among the plurality of subframes, the third subframe on which neither the first pattern image nor the second pattern image is superimposed is an image different from the first subframe and the second subframe. The image processing method according to claim 1. [Claim 6] Both the first pattern image and the second pattern image are superimposed on a video signal with a blue component. The image processing method according to any one of claims 1 to 5. [Claim 7] Based on the pixel values ​​of the video signal in the frame, it is determined whether or not to output the first superimposed subframe and the second superimposed subframe to the display surface. The image processing method according to any one of claims 1 to 5. [Claim 8] There are multiple projection devices that project images onto the display surface, When projecting images onto the display surface from each of the multiple projection devices to project a composite image onto the display surface, the process of outputting the first superimposed subframe and the second superimposed subframe to the display surface is performed alternately by the multiple projection devices. The image processing method according to any one of claims 1 to 5. [Claim 9] One or more processors, The video processing method described in any one of claims 1 to 5 is performed. program. [Claim 10] A first acquisition unit that acquires three or more subframes by dividing the frame contained in the video data in time, An output unit that outputs a first superimposed subframe, which is obtained by superimposing a first pattern image onto a first subframe based on the plurality of subframes, and a second superimposed subframe, which is obtained by superimposing a second pattern image, which is obtained by inverting the pixel values ​​of the first pattern image onto a second subframe based on the plurality of subframes, so as to display on the display surface. A second acquisition unit that acquires the first superimposed subframe and the second superimposed subframe displayed on the display surface by imaging, A third acquisition unit acquires a third pattern image from the difference between the acquired first superimposed subframe and the acquired second superimposed subframe, The system includes a detection unit that detects a shift in the display position of the image projected onto the display surface by comparing the feature points of the acquired third pattern image with reference feature points, Each of the aforementioned subframes is an image composed of subpixels of the same phase in each pixel of the frame. Video processing system.

Images