Determination method, determination device, and program

Abstract

In detecting the pattern image projected from the projection device 20 onto the projection surface SC, unnecessary failure detection is avoided without increasing processing time. [Solution] The information processing device 10, which functions as a determination device according to the present disclosure, comprises a projection control unit 140a, an imaging control unit 140b, a detection unit 140c, and a determination unit 140d. The projection control unit 140a causes the projection device 20 to project a first pattern image, which includes a plurality of first figures arranged in a matrix, onto the projection surface SC. The imaging control unit 140b acquires a first image of the first pattern image captured by the camera 120. The detection unit 140c detects a second pattern image, which includes a plurality of second figures that correspond one-to-one with the plurality of first figures, based on the first image captured. The determination unit 140d determines whether the detection of the first pattern image failed based on the number of figures belonging to the internal pattern among the plurality of second figures.

Description

[[Technical Field]]

[0001] The present disclosure relates to a determination method, a determination apparatus, and a program. [[Background Art]]

[0002] Wall surfaces of residential living rooms often have fine depressions or protrusions, and are not necessarily flat. When an image is projected from a projector onto such a wall surface with fine depressions or protrusions used as a projection surface, distortion corresponding to the depressions or protrusions of the wall surface may occur in the image displayed on the wall surface. In order to avoid such distortion, projectors are sometimes equipped with a correction function that projects a pattern image such as a dot pattern from the projector onto the projection surface, detects an image corresponding to the pattern image from a captured image obtained by capturing the pattern image displayed on the projection surface with a camera, and performs geometric correction or the like on the projected image to cancel the distortion. A dot pattern is a pattern in which dot figures (dot images) are arranged, for example, in a matrix.

[0003] Various methods for detecting a pattern image projected from a projector onto a projection surface have been conventionally proposed, and one example is the method disclosed in Patent Document 1. In the detection method disclosed in Patent Document 1, first, a pattern image (hereinafter referred to as a projected pattern image) is displayed on a projection surface by projecting image light from the projector, and a captured image is obtained by capturing the projected pattern image displayed on the projection surface with a camera. Next, processing for detecting a pattern image corresponding to the projected pattern image (hereinafter referred to as a displayed pattern image) from the captured image is executed. In the detection method disclosed in Patent Document 1, when 90% or more of the plurality of dot figures in the displayed pattern image are successfully detected, it is determined that the displayed pattern image has been successfully detected. If the detection of the displayed pattern image fails, the displayed pattern image is detected from an image obtained by subjecting the captured image to correction processing that increases the possibility of successful detection of the displayed pattern image. [[Prior Art Documents]] [[Patent Documents]]

[0004] [Patent Document 1] Japanese Patent Publication No. 2024-18681 [Overview of the project] [Problems that the invention aims to solve]

[0005] When using a dot pattern image as a projection pattern image, one factor that can cause failure in detecting dot shapes is, for example, that the dot shapes included in the projection pattern image overlap with the frame of the projection surface. When a dot pattern image is used as a projection pattern image and the projection surface has a frame, the technology disclosed in Patent Document 1 determines that detection of the display pattern image has failed if the number of successfully detected dot shapes falls below 90% of the total number of dot shapes, due to dot shapes in the projection pattern image overlapping with the frame of the projection surface, etc. However, if the detection of dot shapes other than those in the peripheral areas of the display pattern image is successful, distortion correction may be performed without problems even if the number of successfully detected dot shapes is less than 90%. In other words, the technology disclosed in Patent Document 1 had the problem that it could unnecessarily determine failure. [Means for solving the problem]

[0006] One aspect of the determination method of the present disclosure is characterized by: acquiring a first captured image by a camera of a first pattern image which includes a plurality of first figures projected from a projector onto a screen and arranged in a first direction and a second direction different from the first direction; detecting a second pattern image corresponding to the first pattern image which includes a plurality of second figures that correspond one-to-one with the plurality of first figures, based on the first captured image; and determining whether the detection of the first pattern image has failed based on the number of at least one fourth figure among the plurality of second figures which is located closer to the center of the second pattern image than at least one third figure.

[0007] Furthermore, one embodiment of the determination device of the present disclosure comprises a communication device that communicates with a projector and a processing device, wherein the processing device acquires a first image captured by a camera of a first pattern image that includes a plurality of first figures projected from the projector onto a screen and arranged in a first direction and a second direction different from the first direction; detects a second pattern image corresponding to the first pattern image, which includes a plurality of second figures that correspond one-to-one with the plurality of first figures, based on the first image; and determines whether the detection of the first pattern image has failed based on the number of at least one fourth figure among the plurality of second figures that is located closer to the center of the second pattern image than at least one third figure.

[0008] One aspect of the program of this disclosure is characterized in that it causes a computer to perform the following actions: acquire a first captured image taken by a camera of a first pattern image which includes a plurality of first figures projected from a projector onto a screen and arranged in a first direction and a second direction different from the first direction; detect a second pattern image corresponding to the first pattern image which includes a plurality of second figures that correspond one-to-one with the plurality of first figures, based on the first captured image; and determine whether the detection of the first pattern image has failed based on the number of at least one fourth figure among the plurality of second figures which is located closer to the center of the second pattern image than at least one third figure. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of a projection system 1 including an information processing device 10 according to one embodiment of the determination device of the present disclosure. [Figure 2] This figure shows an example of a projection surface SC in which the image GA is being projected from the projection device 20. [Figure 3] This figure shows an example of a pattern image PT1 projected from the projection device 20 onto the projection surface SC. [Figure 4] This figure shows an example configuration of the information processing device 10. [Figure 5] This figure shows an example of dot pattern classification in this embodiment. [Figure 6] This flowchart shows the processing flow in the determination method executed by the processing unit 140 according to program PR1. [Figure 7] This figure illustrates the pattern images PT2, PT3, and PT4 of modified example (1). [Modes for carrying out the invention]

[0010] The embodiments described below are subject to various technically preferred limitations. However, the embodiments of this disclosure are not limited to those described below. A. Embodiment Figure 1 shows an example configuration of a projection system 1 including an information processing device 10 according to one embodiment of the determination device of the present disclosure. As shown in Figure 1, the projection system 1 includes a projection device 20, which is, for example, a projector, in addition to the information processing device 10. The projection device 20 includes a display panel on which an image is drawn and an optical system that guides the image light emitted from the display panel to the projection surface SC. In Figure 1, the display panel and the optical system are not shown. The projection device 20 displays an image on the projection surface SC by projecting image light onto the projection surface SC under the control of the information processing device 10. In this embodiment, the projection surface SC is provided on the wall surface of the room in which the projection device 20 is installed. The projection surface SC is an example of a screen in the present disclosure.

[0011] The wall surface of the room where the projection device 20 is installed has small indentations or protrusions. Therefore, even if the outline of the image projected from the projection device 20 onto the projection surface SC is rectangular, as shown in Figure 2, the image GA displayed on the projection surface SC by projecting image light from the projection device 20 will be distorted according to the small indentations or protrusions on the wall surface where the projection surface SC is located. In Figure 2, the symbol W refers to the frame that demarcates the projection surface SC.

[0012] The projection device 20 is equipped with a correction function to correct distortions that occur in the displayed image in accordance with fine depressions or protrusions on the projection surface SC. The projection device 20 displays the pattern image PT1 shown in Figure 3 on the projection surface SC under the control of the information processing device 10. As shown in Figure 3, the pattern image PT1 in this embodiment is a dot pattern image that includes a plurality of dot figures DT arranged in a matrix in the X direction and in the Y direction which is different from the X direction. In this embodiment, the X direction is the direction corresponding to the horizontal scanning direction of the display panel in the projection device 20, and the Y direction is the direction corresponding to the vertical scanning direction of the display panel in the projection device 20. The pattern image PT1 is an example of a first pattern image in this disclosure. The X direction is an example of a first direction in this disclosure, and the Y direction is an example of a second direction in this disclosure. The dot figures DT are an example of a first figure in this disclosure.

[0013] In the following, multiple dot shapes DT arranged in the X direction and in the Y direction different from the X direction in the pattern image PT1 are classified into dot shapes DT1 belonging to the outer perimeter pattern and dot shapes DT2 belonging to the inner pattern. In Figure 3, dots DT1 belonging to the outer perimeter pattern are shown as white circles, and dots DT2 belonging to the inner pattern are shown as black circles. As shown in Figure 3, the outer perimeter pattern refers to a set of dot shapes arranged in N columns (N is an integer of 1 or more) in the X and Y directions along the outer perimeter of a matrix composed of multiple dot shapes. In this embodiment, N=1, but N may be 2, or 3 or more. As shown in Figure 3, the outer perimeter pattern is arranged in a ring along the outer perimeter of the pattern image PT1. Dot shapes DT1 belonging to the outer perimeter pattern are an example of the third shape in this disclosure. The inner pattern refers to a set of dot shapes arranged inside the outer perimeter pattern (i.e., closer to the center of the pattern image PT1 than the dot shapes belonging to the outer perimeter pattern). The dot figure DT2 belonging to the internal pattern is an example of a fourth figure in this disclosure. Since the internal pattern is located inside the outer pattern, the fourth figure is located closer to the center of the pattern image PT1 than at least one third figure.

[0014] In this embodiment, based on the captured image obtained by capturing the pattern image PT1 displayed on the projection surface SC with a camera, various correction parameters for correcting the distortion are determined by the information processing device 10, and these determined correction parameters are set in the projection device 20 by the information processing device 10.

[0015] The information processing device 10 is, for example, a smartphone that communicates with the projection device 20 via a wireless LAN (Local Area Network) or the like. FIG. 4 is a diagram showing a configuration example of the information processing device 10. As shown in FIG. 4, the information processing device 10 includes a communication device 100, a UI (User Interface) device 110, a camera 120, a storage device 130, and a processing device 140. Each of the communication device 100, the UI device 110, the camera 120, and the storage device 130 is connected to the processing device 140 via a bus (not shown in FIG. 4). Note that the camera 120 may be a device separate from the information processing device 10 (for example, a camera connected to the information processing device 10 via a wireless LAN).

[0016] The communication device 100 includes an antenna for transmitting and receiving signals related to wireless communication, and a communication interface circuit that performs encoding and decoding of the signal. The communication device 100 delivers data received from the projection device 20 via a wireless LAN or the like to the processing device 140, and transmits data provided from the processing device 140 to the projection device 20.

[0017] The UI device 110 includes, for example, an input device provided with a plurality of operators such as a numeric keypad, and a display device for displaying various images under the control of the processing device 140. A specific example of the display device is a liquid crystal display. In FIG. 4, illustrations of the input device and the display device are omitted. When an operation such as pressing any operator is performed on the input device included in the UI device 110, the UI device 110 delivers data corresponding to the user's operation content to the processing device 140. Thereby, the content of the user's operation is transmitted to the processing device 140.

[0018] The processing device 140 includes one or more processors. The processor includes, for example, a CPU (Central Processing Unit) and the like. The processing device 140 is an example of the computer in the present disclosure. Although details will be described later, the processing device 140 functions as a control center of the information processing device 10 by operating in accordance with a program PR1 stored in advance in the storage device 130.

[0019] The camera 120 includes a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. In this embodiment, the camera 120 images an imaging area under the control of the processing device 140. The camera 120 outputs image data representing the captured image to the processing device 140.

[0020] The storage device 130 includes a non-volatile memory such as a flash ROM (Read Only Memory), and a volatile memory such as a RAM (Random Access Memory). The non-volatile memory of the storage device 130 stores a program PR1 for causing the processing device 140 to function as the control center of the information processing device 10, pattern image data D1 representing a pattern image PT1, and conversion data D2. The conversion data D2 is data for mutually converting between a position on a captured image captured by the camera 120 and a position on an image projected by a projection device 20. The volatile memory of the storage device 130 is used by the processing device 140 as a work area when executing the program PR1. When an operation instructing execution of the program PR1 is performed on the UI device 110, the processing device 140 reads the program PR1 from the non-volatile memory to the volatile memory, and starts executing the program PR1 read into the volatile memory.

[0021] The processing device 140 operating in accordance with the program PR1 functions as a projection control unit 140a, an imaging control unit 140b, a detection unit 140c, a determination unit 140d, and a calculation unit 140e.

[0022] The projection control unit 140a controls the projection device 20 to project the pattern image PT1 onto the projection surface SC. In this embodiment, the projection control unit 140a reads the pattern image data D1 from the storage device 130 and transmits the pattern image data D1 and a command to project the image represented by the pattern image data D1 to the projection device 20 using the communication device 100, thereby causing the projection device 20 to project the pattern image PT1 onto the projection surface SC.

[0023] The imaging control unit 140b controls the camera 120 to cause the camera 120 to image the projection surface SC on which the pattern image PT1 is displayed. For example, the imaging control unit 140b displays a message on the display device of the UI device 110 instructing the user to position the information processing device 10 so that the entire projection surface SC is included in the imaging area of ​​the camera 120 and to perform the imaging operation. The user of the projection system 1 adjusts the position and orientation of the information processing device 10 relative to the projection surface SC in accordance with the message and then performs the imaging operation. Triggered by the execution of this imaging operation, the imaging control unit 140b causes the camera 120 to take an image and acquires image data representing the image captured by the camera 120 from the camera 120. Thus, the imaging control unit 140b acquires image data representing the image obtained by imaging the projection surface SC on which the pattern image PT1 is displayed. The image obtained by imaging the projection surface SC on which the pattern image PT1 is displayed is an example of the first image in this disclosure.

[0024] The detection unit 140c analyzes the captured image data acquired by the imaging control unit 140b to detect a pattern image corresponding to the pattern image PT1 in the captured image data. The method of detecting the pattern image by the detection unit 140c is the same as the method in Patent Document 1. The pattern image corresponding to the pattern image PT1 in the captured image data includes multiple dot figures that correspond one-to-one with multiple dot figures in the pattern image PT1. The pattern image corresponding to the pattern image PT1 in the captured image is an example of the second pattern image in this disclosure. The dot figures in the second pattern image (multiple dot figures that correspond one-to-one with multiple dot figures in the pattern image PT1) are an example of the second figure in this disclosure.

[0025] The determination unit 140d determines whether the detection of pattern image PT1 has failed based on the number of dot figures belonging to the internal pattern whose position has been successfully detected among the multiple dot figures in the second pattern image. For example, if the number of dot figures belonging to the internal pattern whose position has been successfully detected is equal to or greater than a threshold determined according to the total number of dot figures belonging to the internal pattern (for example, the total number × 0.9), the determination unit 140d determines that the detection of pattern image PT1 has been successful. Conversely, if the number of dot figures belonging to the internal pattern whose position has been successfully detected is less than the above threshold, the determination unit 140d determines that the detection of pattern image PT1 has failed.

[0026] Figure 5 shows an example of classification based on the position detection results for multiple dot shapes in the second pattern image. In Figure 5, black filled circles represent dot shapes belonging to the inner pattern, white circles with solid outlines represent dot shapes belonging to the outer pattern whose position detection was successful, and white circles with dotted outlines represent dot shapes belonging to the outer pattern whose position detection failed. When multiple dot shapes in the second pattern image are classified as shown in Figure 5, the determination unit 140d determines whether or not detection of the pattern image PT1 failed based on the number of dot shapes corresponding to the black filled circles.

[0027] The calculation unit 140e calculates correction parameters for distortion correction for each position of the first figure, based on the position of the dot figure in the second pattern image of the internal pattern whose position has been successfully detected and the position of the dot figure (i.e., the first figure) in the pattern image PT1 corresponding to the dot figure. For example, for a dot figure belonging to an internal pattern whose position has been successfully detected, the calculation unit 140e determines the position on the second pattern image where the dot figure should be located, based on the position of the dot figure (i.e., the first figure) in the pattern image PT1 corresponding to the dot figure and the conversion data D2. Next, the calculation unit 140e calculates the difference (i.e., distortion) between this position and the position in the second pattern image that was actually captured, and calculates correction parameters to cancel out the distortion. Then, the calculation unit 140e sets the calculated correction parameters in the projection device 20 by transmitting them to the projection device 20 via the communication device 100. Furthermore, if the determination unit 140d determines that it has failed to detect the pattern image PT1, the processing unit 140 displays a message on the display device prompting the user to adjust the position of the projection device 20 relative to the projection surface SC and then perform the projection and imaging of the pattern image PT1 again.

[0028] Furthermore, the calculation unit 140e may estimate the position of a dot figure (hereinafter referred to as the fifth figure) belonging to the outer peripheral pattern and for which position detection failed, based on its positional relationship with a fourth figure located around it and for which position detection was successful, and calculate correction parameters based on this estimation result. Specifically, the calculation unit 140e calculates the aforementioned difference for the fourth figure located around the fifth figure and for which position detection was successful, and estimates the position of the fifth figure on the second pattern image based on this difference. According to this embodiment, since the position of the fifth figure is estimated based on its positional relationship with a fourth figure located around it, the accuracy of image correction in the projection device 20 can be improved compared to when the correction is performed with the fifth figure missing. In addition, if there is a dot figure (hereinafter referred to as the sixth figure) of the outer peripheral pattern located further outside the fifth figure for which position detection was successful, the calculation unit 140e may estimate the position of the fifth figure based on the sixth figure and the fourth figure. According to this embodiment, the accuracy of estimating the position of the fifth figure is improved compared to an embodiment in which the position of the fifth figure is estimated based only on the positional relationship with the fourth figures located around the fifth figure. However, the estimation of the position of the fifth figure is not limited to the procedure described above. For example, the calculation unit 140e calculates a transformation matrix that associates the coordinates of each of the four fourth figures in the coordinate system of the captured image with the coordinates of each of the four fourth figures in the coordinate system of the display panel, based on the four fourth figures located around the fifth figure whose positions have been successfully detected. The calculation unit 140e may then estimate the position of the fifth figure in the coordinate system of the captured image based on the transformation matrix and the coordinates of the fifth figure in the coordinate system of the display panel.

[0029] Furthermore, the processing unit 140, which operates according to program PR1, executes a determination method that prominently demonstrates the features of this disclosure. Figure 6 is a flowchart showing the processing flow in this determination method.

[0030] In step SA110, the processing unit 140 functions as a projection control unit 140a. In step SA110, the processing unit 140 reads image data representing the pattern image PT1 from the storage device 130 and transmits the image data and a command instructing the projection of the image represented by the image data to the projection device 20 using the communication device 100, thereby causing the projection device 20 to project the pattern image PT1 onto the projection surface SC. Step SA110 may be omitted. Specifically, instead of the processing unit 140 transmitting a command to the projection device 20 to project the pattern image PT1 onto the projection surface SC, the pattern image PT1 may be projected onto the projection surface SC in response to user operation on the projection device 20, which is independent of control by the processing unit 140. User operation on the projection device 20 may include, for example, operation on operation buttons of the projection device 20 (not shown) or operation on a menu screen that the projection device 20 projects onto the projection surface SC. In this case, the image data representing the pattern image PT1 may be stored in a storage device provided by the projection device 20, rather than in the storage device 130.

[0031] In step SA120, which is executed following step SA110, the processing unit 140 functions as an imaging control unit 140b. In step SA120, the processing unit 140 controls the camera 120 to cause the camera 120 to image the projection surface SC on which the pattern image PT1 is displayed, and acquires image data representing the image from the camera 120.

[0032] In step SA130, which follows step SA120, the processing unit 140 functions as a detection unit 140c. In step SA130, the processing unit 140 analyzes the image data acquired in step SA120 and detects a pattern image corresponding to the pattern image PT1 in the image data represented by the image data.

[0033] In steps SA130 and SA140, the processing unit 140 functions as a determination unit 140d. Based on the number of dot figures belonging to the internal pattern whose position was successfully detected among multiple dot figures in the second pattern image, it determines whether the detection of pattern image PT1 has failed. If the determination result of step SA140 is "No", that is, if it is determined that the detection of pattern image PT1 has been successful, the processing unit 140 executes the process of step SA150. Conversely, if the determination result of step SA140 is "Yes", that is, if it is determined that the detection of pattern image PT1 has failed, the processing unit 140 executes the process of step SA160.

[0034] In step SA160, the processing unit 140 displays a message prompting the user to adjust the position of the projection device 20 relative to the projection surface SC and then perform projection and imaging of the pattern image PT1 again. Subsequently, the processing unit 140 repeats the processing from step SA110 onward. In contrast, in step SA150, the processing unit 140 first calculates correction parameters for distortion correction for each position of the first figure, based on the position of the dot figure belonging to the internal pattern that was successfully detected in the second pattern image and the position of the dot figure (i.e., the first figure) in the pattern image PT1 corresponding to the said dot figure. Then, the processing unit 140 sets the calculated correction parameters in the projection device 20 by transmitting them to the projection device 20 via the communication device 100.

[0035] As explained above, according to this embodiment, whether or not the detection of the pattern image PT1 has failed is determined based on the number of dot figures belonging to the internal pattern. Dot figures belonging to the outer peripheral pattern are highly likely to overlap with the frame of the projection surface SC, and if the detection of the pattern image PT1 has failed is determined including the dot figures belonging to the outer peripheral pattern, unnecessary failure determinations will be made. According to this embodiment, unnecessary failure determinations can be avoided compared to the method in which the detection of the pattern image PT1 has failed including the dot figures belonging to the outer peripheral pattern.

[0036] Prior to detecting the pattern image PT1, it is conceivable to avoid unnecessary failure detection by analyzing the image captured by the camera 120 to detect the frame of the projection surface SC in advance, and then excluding the dot figures belonging to the outer peripheral pattern that overlap with the frame to determine whether the pattern image PT1 has been detected. However, in the embodiment where the frame of the projection surface SC is detected prior to detecting the pattern image PT1, the processing load increases compared to this embodiment by the amount of processing required to detect the frame, and the processing time until the success or failure of the pattern image PT1 detection is determined becomes longer than in this embodiment. In other words, according to this embodiment, it is possible to avoid unnecessary failure detection without increasing the processing time.

[0037] B. Transformation The above embodiment can be modified as follows. (1) In the above embodiment, the first figure corresponding to the third figure in the first pattern image (i.e., the outer peripheral pattern) was arranged in a ring along the outer perimeter of the first pattern image. However, as shown in the pattern image PT2 in Figure 7, the outer peripheral pattern may be provided along one side in the X direction of the pattern image, or as shown in the pattern image PT3 in Figure 7, the outer peripheral pattern may be provided along one side in the Y direction of the pattern image. Furthermore, as shown in the pattern image PT4 in Figure 7, the outer peripheral pattern may be provided in an L-shape along one side in the X direction and one side in the Y direction connected to that side of the pattern image. The density of dot figures may be the same in the internal pattern and the outer peripheral pattern, or the density of dot figures may be different in the internal pattern and the outer peripheral pattern. Furthermore, the density of dot figures in the outer peripheral pattern does not need to be uniform within the outer peripheral pattern; the density of dot figures may become sparser as you move away from the center of the pattern image. In the above embodiment, a dot pattern was used as the pattern image PT1, but patterns other than dot patterns, such as checkerboard patterns, may be used. Additionally, the pattern image PT1 may be a random dot pattern.

[0038] (2) In the above embodiment, the processing unit 140 of the information processing device 10, which communicates with the projection device 20 via a wireless LAN, was made to execute the determination method of this disclosure. However, the program PR1 may be installed in the projection device 20, and the computer of the projection device 20 may be made to execute the determination method of this disclosure by operating the computer in accordance with the program PR1. In this case, the camera 120 may be included in the projection device 20, or it may be connected to the projection device 20 via a wireless LAN or the like.

[0039] (3) The first process, which estimates the position of the fifth figure based on the positional relationship with the fourth figure located around the fifth figure, and the second process, which estimates the position of the fifth figure based on the sixth figure and the fourth figure, which are located further away from the fifth figure and whose positions have been successfully detected, may be switched and executed depending on the conditions.Specific examples of conditions for switching between the first and second processes include conditions related to the processing load on the processing unit 140 (for example, whether the processing load is above a predetermined threshold). Specifically, if the processing load on the processing unit 140 is above a threshold, the processing unit 140 should be made to execute the first process, and if the processing load on the processing unit 140 is below a threshold, the processing unit 140 should be made to execute the second process. The second process has a higher processing load than the first process, and this is to avoid placing an excessive load on the processing unit 140.Another specific example of the above conditions is a condition related to the computational performance of the processing unit 140 (whether the computational performance of the processing unit 140 is above a predetermined threshold). Specifically, if the processing unit 140's computational performance is below a threshold, the processing unit 140 should be made to execute the first process; if the processing unit 140's computational performance is above the threshold, the processing unit 140 should be made to execute the second process. Computational performance includes the number of CPU cores and the operating frequency. Another specific example of the above conditions is user-defined settings. Specifically, the information processing unit 10 accepts an operation to select either the execution of the first process or the execution of the second process, and switches between the first and second processes according to the result of that selection. This configuration avoids making the processing unit 140 execute a process that exceeds its computational performance.

[0040] (4) In the above embodiment, one example of a reason for an unnecessary failure determination was given as the case where the dot figures belonging to the outer peripheral pattern are highly likely to overlap with the frame of the projection surface SC. However, the event of an unnecessary failure determination can also occur when the projection surface SC does not have a frame. For example, when the projection surface SC has a pattern, the dot figures belonging to the outer peripheral pattern may overlap with the pattern of the projection surface SC, resulting in an unnecessary failure determination. For this reason, the invention according to the above embodiment is applicable not only when the projection surface SC has a frame.

[0041] (5) In the above embodiment, the projection control unit 140a, imaging control unit 140b, detection unit 140c, determination unit 140d, and calculation unit 140e were software modules. However, at least one of the projection control unit 140a, imaging control unit 140b, detection unit 140c, determination unit 140d, and calculation unit 140e may be a hardware module such as an ASIC (Application Specific Integrated Circuit). Even if at least one of the projection control unit 140a, imaging control unit 140b, detection unit 140c, determination unit 140d, and calculation unit 140e is a hardware module, the same effects as in the above embodiment will be achieved.

[0042] (6) Program PR1 may be manufactured as a standalone product and may be provided for a fee or free of charge. Specific ways of providing Program PR1 include providing it by writing it to a computer-readable recording medium such as flash ROM, or providing it by downloading it via a telecommunications line such as the Internet.

[0043] C. Summary of this disclosure This disclosure is not limited to the embodiments and modifications described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms. The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate. A summary of this disclosure is provided below.

[0044] (Note 1) One aspect of the determination method of this disclosure includes: acquiring a first captured image by a camera of a first pattern image which includes a plurality of first figures projected from a projector onto a screen and arranged in a first direction and a second direction different from the first direction; detecting a second pattern image corresponding to the first pattern image which includes a plurality of second figures which correspond one-to-one with the plurality of first figures, based on the first captured image; and determining whether the detection of the first pattern image has failed based on the number of at least one fourth figure among the plurality of second figures which is located closer to the center of the second pattern image than at least one third figure. According to the determination method of this aspect, since the determination of whether the detection of the first pattern image has failed is based on the number of fourth figures which are located closer to the center of the second pattern image than at least one third figure, it is possible to avoid unnecessary failure determinations compared to when the determination of whether the detection of the first pattern image has failed includes at least one third figure.

[0045] (Note 2) Another aspect of the determination method of this disclosure is the determination method described in (Note 1), which includes estimating the position of a fifth figure among the plurality of second figures that is located further away from the center of the second pattern image than at least one fourth figure and whose position detection failed, based on the positional relationship between the fifth figure and the fourth figures located around it. According to the determination method of this aspect, since the position of the fifth figure is estimated based on the positional relationship between the fifth figure and the fourth figures located around it, the accuracy of image correction in the projector can be improved compared to when the correction is performed with the fifth figure missing.

[0046] (Note 3) A more preferred method of determination is the method described in (Appendix 2), which includes estimating the position of the fifth figure based on the positional relationship with the fourth figure located around the fifth figure, and estimating the position of the fifth figure based on the sixth figure and the fourth figure, which are among the plurality of second figures and are located further away from the center of the second pattern image than the fifth figure and have been successfully detected. According to this embodiment, the accuracy of estimating the position of the fifth figure is improved compared to the embodiment in which the position of the fifth figure is estimated based only on the positional relationship with the fourth figure located around the fifth figure.

[0047] (Note 4) A more preferred embodiment of the determination method is the determination method described in Appendix 1, which includes, depending on the conditions, executing a first process that estimates the position of a fifth figure among the plurality of second figures that is located further away from the center of the second pattern image than at least one fourth figure and whose position detection failed, based on the positional relationship between the fifth figure and the fourth figures located around it, and a second process that estimates the position of the fifth figure among the plurality of second figures that is located further away from the center of the second pattern image than the fifth figure and whose position detection was successful, based on the sixth figure and the fourth figure. According to this embodiment, the feasibility of executing the second process can be optimized.

[0048] (Note 5) A more preferred method for determining the first figure is the method described in any of (Note 1), (Note 2), (Note 3), or (Note 4), wherein the first figure corresponding to the third figure in the first pattern image is arranged in a ring along the outer periphery of the first pattern image.

[0049] (Note 6) One embodiment of the determination device of the present disclosure comprises a communication device that communicates with a projector and a processing device, wherein the processing device acquires a first captured image taken by a camera of a first pattern image which includes a plurality of first figures projected from the projector onto a screen and arranged in a first direction and a second direction different from the first direction; detects a second pattern image corresponding to the first pattern image which includes a plurality of second figures that correspond one-to-one with the plurality of first figures based on the first captured image; and determines whether the detection of the first pattern image has failed based on the number of at least one fourth figure among the plurality of second figures which is located closer to the center of the second pattern image than at least one third figure. According to the determination device of this embodiment, since the determination of whether the detection of the first pattern image has failed is based on the number of fourth figures which are located closer to the center of the second pattern image than at least one third figure, it is possible to suppress unnecessary failure determinations compared to when the determination of whether the detection of the first pattern image has failed includes at least one third figure.

[0050] (Note 7) One aspect of the program of this disclosure causes a computer to perform the following actions: acquire a first captured image captured by a camera of a first pattern image, which includes a plurality of first figures projected from a projector onto a screen and arranged in a first direction and a second direction different from the first direction; detect a second pattern image corresponding to the first pattern image, which includes a plurality of second figures that correspond one-to-one with the plurality of first figures, based on the first captured image; and determine whether the detection of the first pattern image has failed based on the number of at least one fourth figure among the plurality of second figures that is located closer to the center of the second pattern image than at least one third figure. According to the program of this aspect, since the detection of the first pattern image has failed is determined based on the number of fourth figures that are located closer to the center of the second pattern image than at least one third figure, unnecessary failure determinations can be suppressed compared to when the detection of the first pattern image has failed, including at least one third figure. [Explanation of symbols]

[0051] 10... Information processing device, 20... Projection device, 100... Communication device, 110... UI device, 120... Camera, 130... Storage device, 140... Processing device, 140a... Projection control unit, 140b... Imaging control unit, 140c... Detection unit, 140d... Determination unit, 140e... Calculation unit, PR1... Program.

Claims

1. The first pattern image, which includes a plurality of first figures projected from a projector onto a screen and arranged in a first direction and a second direction different from the first direction, is captured by a camera to obtain a first captured image. Based on the first captured image, a second pattern image corresponding to the first pattern image is detected, which includes a plurality of second figures that correspond one-to-one with the plurality of first figures. The detection of the first pattern image is determined based on the number of at least one fourth figure among the plurality of second figures that is located closer to the center of the second pattern image than at least one third figure, A determination method that includes this.

2. This includes estimating the position of a fifth figure, among the plurality of second figures, that is located further away from the center of the second pattern image than at least one fourth figure, and whose position detection failed, based on its positional relationship with the fourth figures located around it. The determination method according to claim 1.

3. The determination method according to claim 2, wherein the estimation based on the positional relationship with the fourth figure located around the fifth figure includes estimating the position of the fifth figure based on the sixth figure and the fourth figure, which are among the plurality of second figures and are located further away from the center of the second pattern image than the fifth figure, and whose position has been successfully detected.

4. The determination method according to claim 1, comprising: a first process of estimating the position of a fifth figure, among the plurality of second figures, that is located further away from the center of the second pattern image than at least one fourth figure and whose position detection failed, based on the positional relationship between the fifth figure and the fourth figures located around it; and a second process of estimating the position of a fifth figure, among the plurality of second figures, that is located further away from the center of the second pattern image than the fifth figure and whose position detection succeeded, based on the sixth figure and the fourth figure, depending on the conditions, the determination method according to claim 1, wherein the determination method includes switching between these two processes as needed.

5. The determination method according to claim 1, wherein the first figure corresponding to the third figure in the first pattern image is arranged in a ring along the outer periphery of the first pattern image.

6. A communication device that communicates with the projector, A processing unit, and The aforementioned processing apparatus is The process involves acquiring a first image captured by a camera of a first pattern image, which includes a plurality of first figures projected from the projector onto a screen and arranged in a first direction and a second direction different from the first direction. Based on the first captured image, a second pattern image corresponding to the first pattern image is detected, which includes a plurality of second figures that correspond one-to-one with the plurality of first figures. The system performs the following steps: determine whether the detection of the first pattern image has failed based on the number of at least one fourth figure among the plurality of second figures that is located closer to the center of the second pattern image than at least one third figure. Judgment device.

7. On the computer, The first pattern image, which includes a plurality of first figures projected from a projector onto a screen and arranged in a first direction and a second direction different from the first direction, is captured by a camera to obtain a first captured image. Based on the first captured image, a second pattern image corresponding to the first pattern image is detected, which includes a plurality of second figures that correspond one-to-one with the plurality of first figures. The detection of the first pattern image is determined based on the number of at least one fourth figure among the plurality of second figures that is located closer to the center of the second pattern image than at least one third figure, A program that executes the command.

Images