Projection image control system, method, and program

Abstract

To provide a projection video control system which is suitable for a laser scanning type projector, a method, and a program.SOLUTION: A pattern optical sequential projection part 201 radiates a plurality of pattern lights previously designed so that each arrangement in a bright region and a dark region is difference in a video imaged projection region as a projection destination, sequentially. A brightness and darkness determination part 202 determines whether or not an irradiation level sensor 102 is positioned at any one of the bright region and the dark region on the basis of an irradiation value detected by each irradiation sensor 102 in each of irradiating the pattern light of which the brightness and darkness arrangement is difference. A position specification part 203 specifies a position of the irradiation sensor 102 on the basis of a determination result that the brightness and darkness arrangement of each pattern light and each irradiation sensor 102 are positioned to any one of the bright region and the dark region. A control part 204 performs various type controls such as a geometric transform of a rectangular shape collection or light reduction correction or the like of a specification region to a video image projected to a projection object 10 by a video image projection part 30 on the basis of a position of each irradiation sensor 102 to be specified.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a projection image control system, method, and program for projecting an image from a projector onto a predetermined area of ​​a projection object with high accuracy, and more particularly to a projection image control system, method, and program suitable for a laser scanning projector. [Background technology]

[0002] In recent years, projectors have become smaller and more powerful, and their usage is expanding. Lasers and LEDs are used as light sources for small projectors. In particular, laser scanning projectors that use a laser light source and a scanning mirror have focus-free characteristics that allow the image to always be in focus, as well as small size and low power consumption, making them suitable for portable ultra-compact projectors.

[0003] Figure 22 shows a typical configuration of a laser scanning projector. The RGB laser beams are combined into a single beam, which is then incident on a MEMS (micro-electromechanical system) and scanned two-dimensionally on a screen to form a two-dimensional image by utilizing the afterimage caused by the eye.

[0004] In an environment where the projector faces the projection object (screen) directly and both are fixed, there is little need to automatically control the projected image. However, when at least one of the projector and the projection object moves, when you want to control the image according to the movement of a moving object within the projection area, or when you want to project the image accurately onto a specific area within the projection area, a projection image control system is required.

[0005] Patent document 1 discloses a technology in which a light transmission hole and an optical sensor are installed in the housing of a projector, and the angle of the optical axis relative to the screen is estimated by detecting the light reflected from the screen with the optical sensor, thereby automatically performing keystone correction.

[0006] Patent document 2 discloses a technology that automatically detects a presenter in front of the screen by installing optical sensors in multiple places on the screen and protects the presenter's eyes by reducing the amount of light projected near the presenter's eyes.

[0007] Patent Document 3 discloses a technique for using a three-dimensional sensor to detect a flat area suitable for a screen in a three-dimensional space and automatically controlling the projection area.

[0008] Patent Document 4 discloses a technique in which a plurality of optical sensors are provided on a screen, and an alignment pattern is projected from a projector onto the optical sensors, thereby accurately aligning a projected image at a predetermined position.

[0009] Patent Document 5 discloses a technology in which a plurality of reflective materials are placed on a screen, the reflected light is captured by a camera, and the movement of the screen is detected to cause the projected image to follow the movement.

[0010] Patent Document 6 discloses a technology in which a plurality of invisible light markers are placed on a moving object, the invisible light markers are detected by a sensor to detect the position of the moving object, and a projected image is made to track the object.

[0011] Patent Document 7 discloses a technique for accurately detecting the position of an optical sensor that is installed to automatically adjust the display brightness of a projected image, and for allowing an image for evaluation to enter the optical sensor with high accuracy. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] JP 2005-159426 A [Patent Document 2] JP 2013-25014 A [Patent Document 3] JP 2017-73705 A [Patent Document 4] Japanese Patent Application Publication No. 03-259188 [Patent Document 5] JP 2019-102931 A [Patent Document 6] JP 2011-254411 A [Patent Document 7] JP 2009-175355 A Summary of the Invention [Problem to be solved by the invention]

[0013] In all of the above conventional techniques, there is a technical problem in that it is difficult to project an image from a scanning projector such as a laser scanning projector onto a predetermined projection target area of ​​a projection target with high accuracy.

[0014] The projection image control systems disclosed in Patent Documents 1 to 3 are all capable of automatically controlling the projected image (keystone correction, dimming of specific areas, projection control onto flat areas), but they are not technologies that can be applied to applications such as high-precision image projection onto a target projection area.

[0015] The projection image control systems disclosed in Patent Documents 4 to 7 aim to project images with high accuracy onto a projection target area or an optical sensor for evaluation, but are premised on application to a projection type projector that enlarges and projects an image displayed on an image display element, etc., using a projection optical system, etc. Therefore, application to a scanning type projector is difficult, and even if it could be applied, the accuracy of projection control would be insufficient.

[0016] Figure 23 is a schematic diagram showing a method for detecting the position of an illuminance sensor using a projector. A spot-shaped bright area is scanned in the pixel coordinate system of the projected image, and the scanning position at the time when the bright area overlaps with the light-receiving surface of the illuminance sensor and the output of the illuminance sensor changes is regarded as the position of the illuminance sensor.

[0017] By making the brightness as high as possible in bright areas and making the brightness as low as possible in dark areas, a high illuminance value is detected when a bright area overlaps the light receiving surface of the illuminance sensor, and a low illuminance value is detected when a dark area overlaps because no projection light enters, making it possible to accurately identify the position of the illuminance sensor.

[0018] In the case of a projection type projector, for example, a binary image (pattern light) in which only a small circular area in the projected image is a bright area and the rest is a dark area is projected from the projector, and the pattern light is controlled so that the position of the bright area changes over time (for example, frame by frame). When the bright area overlaps with the light receiving surface of the illuminance sensor, the output of the illuminance sensor changes significantly as shown in Figure 24(a), so the position of the bright area where the illuminance value reaches its peak can be regarded as the position of the illuminance sensor.

[0019] In contrast, in the case of a scanning projector, even if the beam spot displaying the bright area overlaps with the optical sensor, the projected light does not enter the optical sensor except for the brief period when the scanning of the laser or the like exactly overlaps with the optical sensor, as shown in Fig. 1(b), so there is no strong correlation between the degree of overlap between the pattern light and the optical sensor and the illuminance value. Therefore, if the position where the illuminance value reaches its peak is specified as the position of the optical sensor, there is a problem that the alignment error will become large.

[0020] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a projection image control system, method, and program that can solve the above technical problems and, in particular, project an image from a scanning projector onto a predetermined area of ​​a projection target with high accuracy. [Means for solving the problem]

[0021] In order to achieve the above-mentioned object, the present invention provides a projection image control system in which a plurality of illuminance sensors whose relative positions with respect to a projection target area are provided on a projection target object and projection onto the projection target area is controlled based on the position of each illuminance sensor in a pixel coordinate system.The system is characterized in that it comprises a sequential projection means that sequentially projects a plurality of pattern lights in which bright areas and dark areas are arranged in the pixel coordinate system onto the area in which the illuminance sensors are present while switching at predetermined time intervals, a light / dark discrimination means that discriminates whether the position of each projected pattern light is a bright area or a dark area based on the output of the illuminance sensor at each time interval, and a position identification means that identifies the position of the illuminance sensor based on the result of the discrimination.

[0022] Furthermore, the present invention can be realized not only as a projection image control system having such a characteristic configuration, but also as a projection image control method having a procedure that includes such a characteristic configuration, or as a projection image control program that causes a computer to execute such a procedure. Effect of the Invention

[0023] According to the present invention, it becomes possible to project an image onto a predetermined area of ​​a projection object with high accuracy, particularly from a scanning projector. [Brief description of the drawings]

[0024] [Figure 1] 1 is a functional block diagram showing a configuration of a main part of a projection image control system to which the present invention is applied. [Diagram 2] FIG. 4 is a diagram showing the relationship between a light mask and an illuminance sensor. [Diagram 3] 11 is a diagram showing the relative positional relationship between a projection target area and a plurality of illuminance sensors; FIG. [Figure 4] FIG. 4 is a functional block diagram of a sensor reading unit. [Diagram 5] 1 is a diagram showing the relationship between a pixel coordinate system that controls a projected image and a projection target object. [Figure 6] FIG. 1 is a functional block diagram of a first embodiment of a projection image control system. [Figure 7] 4 is a flowchart showing a position identification procedure in the first embodiment. [Figure 8] 5A to 5C are diagrams illustrating a procedure for identifying a horizontal position in the first embodiment. [Figure 9] FIG. 13 is a diagram showing a method for distinguishing between a bright region and a dark region. [Figure 10] 5A to 5C are diagrams illustrating a procedure for identifying a position in a vertical direction in the first embodiment. [Figure 11] 13A and 13B are diagrams illustrating a position specifying procedure in the second embodiment of the projection image control system. [Figure 12] FIG. 11 is a diagram showing an example of a termination condition. [Figure 13] 13A and 13B are diagrams illustrating a position specifying procedure in the projection image control system according to the third embodiment. [Figure 14] 13A and 13B are diagrams illustrating a position specifying procedure in the projection image control system according to the fourth embodiment. [Figure 15] FIG. 13 is a functional block diagram of a projection image control system according to a fifth embodiment. [Figure 16] 13 is a flowchart showing a position identification procedure in the fifth embodiment. [Figure 17] FIG. 13 is a diagram showing an example of an optical pattern in the fifth embodiment. [Figure 18] FIG. 13 is a diagram illustrating a position specifying procedure according to the fifth embodiment. [Figure 19] FIG. 13 is a functional block diagram of a projection image control system according to a sixth embodiment. [Figure 20] 23 is a flowchart showing a position identification procedure in the sixth embodiment. [Figure 21] FIG. 23 is a diagram showing an example of an optical pattern in the sixth embodiment. [Figure 22] FIG. 1 is a diagram showing a typical configuration of a laser scanning projector. [Diagram 23] 10A and 10B are diagrams illustrating a method for detecting the position of an illuminance sensor. [Figure 24]1A and 1B are diagrams illustrating a method for detecting the position of an illuminance sensor by each of a projection type and a scanning type projector. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Fig. 1 is a functional block diagram showing the configuration of the main parts of a projection image control system 1 to which the present invention is applied, and the main components are a projection object 10, an image projection control unit 20, and an image projection unit 30.

[0026] The projection object 10 is equipped with a plurality of illuminance sensors 102 that detect the illuminance of the illuminance of the illuminance light, an optical mask 101 that blocks a part of the illuminance light incident on the light receiving section of each illuminance sensor 102, and a sensor reading section 103 that reads the sensor signal output by each illuminance sensor 102 and converts it into an illuminance value. Each illuminance sensor 102 is configured integrally with or separately from an image screen or the like (not shown) that visualizes the projection image generated by the illuminance light.

[0027] 2 is a diagram showing the relationship between an optical mask 101 and an illuminance sensor 102, in which only the irradiated light that has passed through the optical mask 101 is received by the illuminance sensor 102. The shape of the optical mask's light-transmitting region (non-masked region) may be a simple shape such as a rectangle or a circle, or may have a complex pattern structure.

[0028] 3 is a diagram showing the relative positional relationship between a projection target area R1 onto which an image for viewing is projected on the projection target object 10 and a plurality of illuminance sensors 102. In this embodiment, the optical mask 101 is provided on the light receiving portion of the illuminance sensor 102, and the position of each optical mask 101 as viewed from the image projection unit 30 corresponds to the position of the corresponding illuminance sensor 102. For convenience, the position of each optical mask 101 may be regarded as the position of each illuminance sensor 102 in the description.

[0029] In this embodiment, the four illuminance sensors 102 and their optical masks 101 are located within the gap area between the image projection area on the image screen where projection by the image projection unit 30 is physically possible and the projection target area R1, and are fixed at a known position relative to the projection target area R1.

[0030] Each illuminance sensor 102 detects light transmitted through the optical mask 101 and outputs a sensor signal such as a current value according to the intensity of the light to the sensor reading unit 103. Each illuminance sensor 102 can be implemented using any device that converts light into a sensor signal and outputs it, generally called a photodiode or phototransistor, or a device such as a photoresistor whose electrical resistance decreases according to the intensity of light.

[0031] The sensor reading unit 103 reads the sensor signals output by each illuminance sensor 102, converts them into illuminance values, and outputs them to the image projection control unit 20. Fig. 4 is a functional block diagram of the sensor reading unit 103, and here, a case will be described where the illuminance sensor 102 is a photodiode as an example.

[0032] The sensor reading unit 103 converts analog sensor signals such as current values ​​at time t output by each illuminance sensor 102 into digital signals, converts them into corresponding illuminance values, corrects errors, and outputs the illuminance values ​​L of each illuminance sensor 102. 1,t ,L 2,t …L K,t (K is the number of illuminance sensors) to the image projection control unit 20.

[0033] In this embodiment, a single optical mask 101 is described as covering the light receiving portion of a single illuminance sensor 102, but a configuration may also be adopted in which multiple illuminance sensors 102 are arranged closely together, all of the light receiving portions of the multiple illuminance sensors 102 are covered with a single large optical mask 101, and light transmitted through the optical mask 101 is detected without omission by the light receiving portions of the multiple illuminance sensors 102. In this case, the sensor reading unit 103 can add up the illuminance values ​​of the multiple illuminance sensors 102 corresponding to one optical mask 101, thereby treating them as if one large illuminance sensor 102 was used. In this embodiment, a configuration may also be adopted in which the optical mask 101 is not used.

[0034] The image projection control unit 20 includes a pattern light sequential projection unit 201, a light / dark discrimination unit 202, a position identification unit 203, and a control unit 204, and is configured to read the illuminance value L of each illuminance sensor 102 at time t obtained from the sensor reading unit 103. 1,t ,L 2,t …L K,t The position of each illuminance sensor 102 is identified based on the position of each illuminance sensor 102. Furthermore, a projection target area R1 is identified based on the position of each illuminance sensor 102, and various controls such as geometric transformation such as keystone correction and dimming correction of a specific area are performed on the image projected by the image projection unit 30 onto the projection target area R1.

[0035] As shown in FIG. 5, when the projection target region R1 of the projection target object 10 is a rectangular region, K (four in this embodiment) illuminance sensors 102 are arranged near the periphery of the rectangular region, and the optical mask 101 is fixedly arranged on the same plane as the projection target region R1, the central coordinate P M,1 ,P M,2 ,P M,3 ,P M,4 and the coordinates P of the four corners that define the projection target area R1 T,1 ,P T,2 ,P T,3 ,P T,4 can be defined in the pixel coordinate system (u, v) of the image projected by the image projection unit 30.

[0036] Then, the center coordinates P M,1 ~P M,4 The coordinates P of the four corners of the projection target area R1 T,1~P T,4 The homography matrix H MT By using the homography matrix H, it is possible to identify the projection target region R1 from the central coordinates of each optical mask 101. Note that the optical mask 101 and the projection target region R1 have a known positional relationship, so MT In other words, the central coordinates P of each optical mask 101 can be calculated in advance. M,1 ,P M,2 ,P M,3 ,P M,4 By specifying the projection target region R1, the projection target region R1 can be specified.

[0037] Therefore, in this embodiment, the position of each optical mask 101 is taken to represent the position of each illuminance sensor 102, the position of each optical mask 101 in the pixel coordinate system is identified, and the position of the projection target region R1 in the pixel coordinate system is identified based on the position of each optical mask 101.

[0038] The masks provided in each illuminance sensor 102 may all have the same unmasked area, or may all have different unmasked areas. The size (area) of the unmasked area can be freely set, but it is desirable to set it to a size smaller than the light receiving portion of the illuminance sensor 102. This allows the illuminance sensor 102 to detect all light that has passed through the optical mask 101.

[0039] The pattern light sequential projection unit 201 sequentially irradiates a plurality of pattern lights that are pre-designed so that the arrangement of light and dark areas differs in the image projection area of ​​the projection destination. Each time pattern light with a different light and dark arrangement is irradiated, the light and dark discrimination unit 202 discriminates whether the illuminance sensor 102 is located in a light area or a dark area based on the illuminance value detected by each illuminance sensor 102.

[0040] The position identification unit 203 identifies the position of the illuminance sensor 102 based on the light and dark arrangement of the projected pattern light and the determination result of whether the position of each illuminance sensor 102 is in a light region or a dark region. The control unit 204 performs various controls, such as geometric transformation such as keystone correction and dimming correction of a specific region, on the image projected by the image projection unit 30 onto the projection object 10, based on the identified position of each illuminance sensor 102.

[0041] Such an image projection control unit 20 can be configured by implementing applications (programs) that realize the functions described below in detail on a general-purpose computer or server equipped with a CPU, ROM, RAM, bus, interface, etc. Alternatively, it can be configured as a dedicated machine or a single-function machine in which part of the application is implemented as hardware or software.

[0042] The image projection unit 30 projects the corrected image data onto a video screen to visualize it. In this embodiment, the image projection unit 30 is described as being a laser scanning projector device, but the present invention can also be applied to a projection type projector device.

[0043] 6 is a functional block diagram showing the configuration of a first embodiment of the projection image control system 1, where the same reference numerals as those described above denote the same or equivalent parts. This embodiment is characterized in that the position identification unit 203 of the image projection control unit 20 includes an existence area narrowing down unit 203a, and identifies the position by repeatedly narrowing down the existence area of ​​each illuminance sensor 102 for each irradiation of pattern light.

[0044] FIG. 7 is a flowchart showing a procedure for identifying the position of each illuminance sensor 102 in the first embodiment, and is repeated for each illuminance sensor 102 whose position is to be identified.

[0045] In step S1, one of the illuminance sensors 102 is selected to identify the position. In step S2, a search range is set in the presence area of ​​the selected illuminance sensor 102. Here, the search range is set (initialized) to the entire image projection area. In step S3, the pattern light sequential projection unit 201 irradiates pattern light in a light and dark arrangement that divides the presence area into two equal light and dark areas.

[0046] In this embodiment, in order to first identify the position in the horizontal direction (u direction) of the pixel coordinate system of the illuminance sensor 102, a pattern light with a light and dark arrangement in which the left half of the existence area is a light area and the right half is a dark area is first irradiated, as shown in FIG. 8(a).

[0047] In step S4, the light / dark discrimination unit 202 discriminates whether the position is a light area or a dark area based on the illuminance value of the selected illuminance sensor 102. In this embodiment, as shown in FIG. 9, a predetermined time (transmitted light presence / absence determination period) T c The laser light is scanned within the image projection area to project the pattern light.

[0048] As shown in FIG. 1(a), the timing of switching the light patterns is t s Transmitted light presence / absence determination period T c Even if the illuminance value of the illuminance sensor 102 exceeds the predetermined threshold value τ L If the value is equal to or greater than this, it is determined that the pixel is located in a bright region. c Within the threshold τ L If the above conditions are not satisfied, it is determined that the sensor is located in a dark area. In the illustrated example, the position of the illuminance sensor 102 is determined to be a bright area.

[0049] The method for distinguishing between light and dark is as described above. s From s +T c Within the period, the illuminance value exceeds the threshold value τ L "t" is not limited to "more than one time" s From s +T c Within the period, the illuminance value exceeds the threshold value τ L"Exceeds a certain number of times" or "t s From s +T c Within the period, the illuminance value exceeds the threshold value τ L The total time that the threshold value is τ T "Exceeds" and "t s From s +T c The sum of the illuminance values ​​during the period is equal to a given threshold τ L_total "Exceeds" is also acceptable.

[0050] Illuminance value threshold τ L must be set to a value that is greater than the illuminance value when no pattern light is incident on the illuminance sensor 102 and less than the illuminance value when the pattern light is incident. This value may be specified by a user input, or may be set automatically. If it is set automatically, for example, the entire image projection area is treated as a bright area and a predetermined time T A (T A >T C ) is projected, the maximum illuminance value (minimum, average, median, etc. are also acceptable) is L max , the entire image projection area is darkened for a predetermined time T A The maximum illuminance value (minimum, average, median, etc.) detected when projecting only L min And L max and L min It can be automatically set as an intermediate value between

[0051] In step S5, the existence area narrowing unit 203a compares the result of the light / dark discrimination with the light / dark arrangement of the light pattern to narrow the existence area of ​​the illuminance sensor 102 to the left half of the search range. In step S6, the search range is updated to the narrowed existence area.

[0052] In step S7, it is determined whether a predetermined termination condition is satisfied. In this embodiment, an upper limit is set for the number of times the existence region is narrowed down, and the termination condition can be when the number of times the existence region is narrowed down reaches the upper limit.

[0053] If the termination condition is not satisfied, the process returns to step S3, and as shown in Fig. 8(b), the narrowed existence area is set as a search range, and a light pattern with a light and dark arrangement is irradiated, in which the left half is a light area and the other area including the right half is a dark area. In steps S4 to S6, the existence area is narrowed down to either a light area or a dark area based on the illuminance value of the illuminance sensor 102, and the search range is updated. In the illustrated example, the existence area is narrowed down to a dark area.

[0054] If it is determined in step S7 that the termination condition is not satisfied, the process returns to step S3, and as shown in Fig. 8(c), the narrowed existence area is used as a search range, and a light pattern with a light and dark arrangement is projected, with the left half of the area being a light area and the other areas including the right half being a dark area. In steps S4 to S6, the existence area is narrowed down to either a light area or a dark area based on the illuminance value of the illuminance sensor 102, and the search range is updated. This narrowing down of the existence area is repeated until it is determined in step S7 that the termination condition is satisfied.

[0055] If the termination condition is met, the process proceeds to step S8, where the horizontal coordinate of the most recently narrowed-down existence area or the horizontal center coordinate of that existence area is regarded as the horizontal position of that illuminance sensor 102. In step S9, it is determined whether or not horizontal position determination has been completed for all illuminance sensors 102. If not, the process returns to step S1, where another illuminance sensor 102 whose position is to be determined is selected and the above processes are repeated.

[0056] Once the horizontal positions of all illuminance sensors 102 have been identified in this manner, as shown in FIG. 10, the vertical position (v direction) of the pixel coordinate system of the illuminance sensors 102 is then identified by first narrowing down the area of ​​presence by projecting a light pattern with a light and dark arrangement in which the upper half of the vertical direction is a light area and the lower half is a dark area, as shown in FIG. 10(a).

[0057] Once the existence area is narrowed down to a dark area, as shown in the same figure (b), the search range is updated to the narrowed existence area, and the existence area is further narrowed down by projecting a light pattern with a light and dark arrangement in which the upper half is a light area and the other areas including the lower half are dark areas.

[0058] When the existence region is narrowed down to a dark region, as shown in Fig. 1C, the search range is updated to the narrowed existence region, and the existence region is further narrowed down by projecting a light pattern with a light and dark arrangement in which the upper half is a light region and the other regions including the lower half are dark regions. This narrowing down of the existence region is also repeated until a predetermined end condition is met.

[0059] When a predetermined termination condition is met, the process proceeds to step S8, where the vertical coordinate of the most recently narrowed-down existence area or the vertical center coordinate of that existence area is regarded as the vertical position of that illuminance sensor 102. In step S9, it is determined whether or not vertical position determination has been completed for all illuminance sensors 102. If not, the process returns to step S1, where another illuminance sensor 102 whose position is to be determined is selected and the above processes are repeated.

[0060] According to this embodiment, in the projection image control system 1 employing a scanning projector, even if the projector does not directly face the projection target (screen), it is possible to accurately project an image onto the projection target area R1 with high precision.

[0061] In the above first embodiment, it has been described that the illuminance sensor 102 is necessarily present within the search range, and if the selected illuminance sensor 102 does not output an illuminance value corresponding to a bright area, it is deemed to be present in a dark area. However, the present invention is not limited to this, and as in the second embodiment shown in Fig. 11, a pattern light [Fig. 11 (a'), (b'), (c')] in a light and dark arrangement in which the bright area and the dark area are interchanged may also be irradiated, and only the illuminance sensor 102 that outputs an illuminance value exceeding a threshold value in a period corresponding to a bright area may be determined to be located in a bright area in the light and dark arrangement.

[0062] Furthermore, in the second embodiment described above, it is assumed that the illuminance sensor 102 is always present within the search range. Therefore, even when either pattern of light with a reversed light and dark arrangement is irradiated, the threshold value τ L If an illuminance value exceeding this value is detected, it can be estimated that the position of the illuminance sensor 102 in question is on the boundary line between the areas, as shown in FIG.

[0063] Therefore, it is possible to add to the termination conditions that the presence area is determined to be a bright area regardless of whether a pattern light with a reversed light / dark arrangement is irradiated, and immediately identify the position of the illuminance sensor 102 as being on the boundary line of each area when the termination upper limit is met.

[0064] 13 is a diagram showing a schematic diagram of a procedure for identifying the position of the illuminance sensor 102 according to the third embodiment of the present invention. In the above-described first and second embodiments, the horizontal (or vertical) position is identified first, and then the vertical (or horizontal) position is identified, but the present embodiment is characterized in that the presence area in the horizontal and vertical directions is narrowed down alternately.

[0065] In this embodiment, the image projection area is first set as the search range, and the area where the illuminance sensor 102 exists is determined by irradiating it with a pattern light having a light area in the upper half and a dark area in the lower half as shown in Fig. 1(a). In the illustrated example, the area where the illuminance sensor 102 exists is determined to be the lower half of the search range.

[0066] Next, as shown in Fig. 1(b), the image projection area is again used as the search range, and this time the presence area of ​​the illuminance sensor is determined by irradiating it with a light pattern arrangement in which the left half in the horizontal direction is a light area and the right half is a dark area. In the example shown, the presence area is determined to be the right half of the search range.

[0067] In this embodiment, the presence area can be narrowed down to the lower left quarter of the image projection area by the above two-step procedure, so the search range is then updated to that quarter, and the presence area of ​​the illuminance sensor is determined by irradiating it with a light pattern arrangement in which the upper half is a light area and the lower half is a dark area, as shown in Fig. 1(c). In the example shown, the presence area is determined to be the lower half of the search range.

[0068] Next, as shown in Fig. 1(d), the 1 / 4 region is set as the search range, and a light pattern arrangement in which the left half is a light region and the right half is a dark region is irradiated to determine the presence region of the illuminance sensor. In the example shown, the presence region is determined to be the right half of the search range.

[0069] In this embodiment, such narrowing down of the existence area is alternately repeated until a predetermined termination condition is met, and the position of the illuminance sensor 102 is identified based on the horizontal and vertical coordinates of the existence area at the time the termination condition is met.

[0070] 14 is a diagram showing a schematic diagram of a procedure for identifying the position of an illuminance sensor according to a fourth embodiment of the present invention. In the above embodiments, the positions of the illuminance sensors 102 are identified one by one, but this embodiment is characterized in that it is possible to identify the positions of a plurality of illuminance sensors 102 simultaneously.

[0071] In this embodiment, first, the entire image projection area is set as the search range, and pattern light with a light and dark arrangement in which the left half is a light area and the right half is a dark area is irradiated as shown in Fig. 1(a).Furthermore, as shown in Fig. 1(a'), pattern light with the light and dark arrangement reversed, that is, pattern light with the left half being a dark area and the right half being a light area, is irradiated to determine the presence area of ​​the two illuminance sensors 102.

[0072] In this embodiment, the presence area of ​​the illuminance sensor 102a is narrowed down to the left half by irradiating it with the pattern light of the light and dark arrangement of FIG. 1(a), and the presence area of ​​the illuminance sensor 102b is narrowed down to the right half by irradiating it with the pattern light of the light and dark arrangement of FIG.

[0073] Next, the narrowed-down existence area of ​​each of the illuminance sensors 102a and 102b is treated as a new search range, and light is irradiated with a light-dark pattern arrangement in which the left half is a light area and the right half is a dark area, as shown in Fig. 1(b).Furthermore, light is irradiated with a light-dark pattern arrangement in the opposite direction, as shown in Fig. 1(b'), to further narrow down the existence area of ​​each of the illuminance sensors 102.

[0074] In this embodiment, the presence area of ​​the illuminance sensor 102a is narrowed down to the right half of the left half of the image projection area by irradiating the pattern light with the light and dark arrangement of FIG. 1(b), and the presence area of ​​the illuminance sensor 102b is narrowed down to the right half of the right half of the image projection area by irradiating the pattern light with the light and dark arrangement of FIG. 1(b').

[0075] Next, the narrowed-down existence area of ​​each of the illuminance sensors 102a and 102b is treated as a new search range, and light is irradiated with a pattern of light and dark arrangement in which the left half is a light area and the other area including the right half is a dark area, as shown in Fig. 1(c).Furthermore, the existence area of ​​each of the illuminance sensors is narrowed down by irradiating light with a pattern of light and dark arrangement inversely, as shown in Fig. 1(c').

[0076] In the illustrated example, the presence area of ​​the illuminance sensor 102a is narrowed down to the left half of the right half of the left half of the image projection area by irradiating the pattern light of the light and dark arrangement of FIG. 1C, and the presence area of ​​the illuminance sensor 102b is narrowed down to the left half of the right half of the right half of the image projection area. Then, the positions of the illuminance sensors 102a and 102b are identified based on the presence areas of the illuminance sensors 102a and 102b at the time when a predetermined termination condition is satisfied. According to this embodiment, the positions of multiple illuminance sensors can be identified simultaneously in one search procedure, so that the time required for position identification can be shortened.

[0077] 15 is a functional block diagram showing the configuration of a projection image control system 1 according to a fifth embodiment of the present invention, where the same reference numerals as those described above denote the same or equivalent parts. This embodiment is characterized in that the light / dark discrimination unit 202 of the image projection control unit 20 includes a light / dark time series recording unit 202a, and the position identification unit 203 identifies the position of each illuminance sensor 102 based on the light / dark time series record.

[0078] In each of the above embodiments, the presence area of ​​the illuminance sensor 102 is identified for each illumination of a pattern light with a different light / dark arrangement, and the presence area is sequentially narrowed down based on the identification result, and the position is finally identified. In contrast, in this embodiment, a plurality of pattern lights with different light / dark arrangements are illuminated sequentially regardless of the presence area of ​​the illuminance sensor 102, and after all the pattern lights are illuminated in a predetermined order, the position of each illuminance sensor 102 is identified based on the time series of the light / dark identification results recorded for each pattern light.

[0079] In this embodiment, the light and dark arrangement of each pattern light is designed in advance so that the time series of light and dark that appears at each position within the search range when each pattern light is irradiated in a predetermined order is unique to each position. Each time a pattern light with a different light and dark arrangement is irradiated, the light and dark time series storage unit 202a records binary information indicating whether the position is a light area or a dark area in time series based on the illuminance value of each illuminance sensor. The position identification unit 203 identifies the position for each illuminance sensor 102 based on the time series record of light and dark.

[0080] FIG. 16 is a flowchart showing a procedure for identifying the position of each illuminance sensor 102 in this embodiment.

[0081] In this embodiment, as shown in an example in Figure 17, the image projection area is virtually divided into n x m (here, 4 x 4) blocks according to the accuracy required for position identification, and a first pattern light [Figure 17(a)] is prepared in which the first and second columns in the left half are dark areas and the third and fourth columns in the right half are bright areas, a second pattern light [Figure 17(b)] is prepared in which the first and third columns are dark areas and the second and fourth columns are bright areas, a third pattern light [Figure 17(c)] is prepared in which the first and second rows in the upper half are dark areas and the third and fourth rows in the lower half are bright areas, and a fourth pattern light [Figure 17(d)] is prepared in which the first and third rows are dark areas and the second and fourth rows are bright areas.

[0082] In step S21, first, a first pattern light is selected. In step S22, the selected pattern light is projected by the pattern light sequential projection unit 201. In step S23, the light and dark based on the illuminance value of each illuminance sensor is discriminated by the light and dark discrimination unit 202, and the discrimination result is recorded in the light and dark time series recording unit 202a, for example, as light region=1 and dark region=0.

[0083] In step S24, it is determined whether recording of the light / dark discrimination results for all of the first through fourth pattern lights has been completed. If not, the process returns to step S21, and the second, third, and fourth pattern lights are selected in that order, and the above-mentioned processes are repeated.

[0084] When recording of the light / dark discrimination results for all of the first through fourth pattern lights is completed, the process proceeds to step S25, where the position of each illuminance sensor is identified based on the time series of light and dark. As shown in Fig. 18, the position of the illuminance sensor 102 whose time series of light and dark corresponding to each of the first through fourth pattern lights is

[0101] is identified as a position in row 2 and column 2. Similarly, the position of the illuminance sensor 102 whose time series of light and dark is

[1110] is identified as a position in row 3 and column 4.

[0085] FIG. 19 is a functional block diagram showing the configuration of a projection image control system 1 according to a sixth embodiment of the present invention, where the same reference numerals as those described above denote the same or equivalent parts.

[0086] This embodiment is characterized in that the light / dark discrimination unit 202 of the image projection control unit 20 is equipped with a light / dark time series recording unit 202a, and the position identification unit 203 is equipped with an existence area narrowing down unit 203a, and the existence area of ​​each illuminance sensor 102 is repeatedly narrowed down based on the light / dark time series.

[0087] FIG. 20 is a flowchart showing a procedure for identifying the position of each illuminance sensor 102 in the sixth embodiment, which is repeated for each illuminance sensor 102.

[0088] In step S31, one of the illuminance sensors 102 for identifying the position is selected. In step S32, a search range is set to the presence area of ​​the selected illuminance sensor 102. Here, the search range is set (initialized) to the entire image projection area. In step S33, as shown in Fig. 21, the search range is virtually divided into four equal parts, and a fifth pattern of light in which the first and second columns are bright areas and the third and fourth columns are dark areas from the left [Fig. 21(a)] and a sixth pattern of light in which the first and third columns are bright areas and the second and fourth columns are dark areas from the left [Fig. 21(a')] are selected in that order.

[0089] In step S34, the fifth pattern light selected first is projected by the pattern light sequential projection unit 201. In step S35, the light and dark based on the illuminance value of the illuminance sensor 102 is discriminated by the light and dark discrimination unit 202, and the discrimination result is recorded in the light and dark time series recording unit 202a, for example, as light region=1 and dark region=0.

[0090] In step S36, it is determined whether recording of the light / dark discrimination results for each pattern light is completed. If not, the process returns to step S33, and an unselected sixth pattern light is selected and the above processes are repeated.

[0091] When recording of the light / dark discrimination results for the two types of pattern light is completed, the process proceeds to step S37, where the presence area of ​​the illuminance sensor 102 is narrowed down based on the time series of light and dark. In the illustrated example, the presence area of ​​the illuminance sensor 102 is discriminated as a "light area" when the fifth pattern light is irradiated, and as a "dark area" when the sixth pattern light is irradiated, the presence area of ​​the illuminance sensor 102 is narrowed down to the second column based on the time series of the discrimination results. In step S38, the next search range is updated to the area in the second column.

[0092] In step S39, it is determined whether a predetermined termination condition is satisfied. If the termination condition is not satisfied, the process returns to step S33, and the above-mentioned processes are repeated for the narrowed-down existence region to further narrow down the existence region, as shown in Fig. 21(b) and (b').

[0093] This narrowing down of the existence area is repeated until a predetermined end condition is met. When the end condition is met, the process proceeds to step S40, where the horizontal coordinate of the latest narrowed down existence area is identified as the horizontal position of illuminance sensor 102. In step S41, it is determined whether position identification has been completed for all illuminance sensors. If not, the process returns to step S31, where another illuminance sensor whose position is to be identified is selected, and the above processes are repeated. When horizontal position identification is completed, a similar process is performed in the vertical direction to identify the position of each illuminance sensor.

[0094] In the above embodiment, it has been described that a pattern light with a light and dark arrangement is irradiated so that all areas other than the search range become dark areas. However, the present invention is not limited to this, and it is also possible to irradiate a pattern light with a light and dark arrangement so that all areas other than the search range become bright areas.

[0095] Furthermore, the above-described embodiment makes it possible to project images with high viewing quality, which makes it possible to contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure and promote inclusive and sustainable industrialization." [Explanation of symbols]

[0096] 1...projection image control system, 10...projection object, 20...image projection control unit, 30...image projection unit, 101...light mask, 102...illuminance sensor, 103...sensor reading unit, 201...pattern light sequential projection unit, 202...light and dark discrimination unit, 202a...light and dark time series recording unit, 203...position identification unit, 203a...existence area narrowing down unit, 204...control unit,

Claims

1. A projection image control system is provided with a plurality of illuminance sensors, the relative positions of which with respect to a projection target area are known, on a projection target, and projection onto the projection target area is controlled based on the position of each illuminance sensor in a pixel coordinate system, a sequential projection means for sequentially projecting a plurality of pattern lights, each having a bright region and a dark region arranged in a pixel coordinate system, onto an area where the illuminance sensor is present while switching the pattern lights at a predetermined time interval; a light / dark discrimination means for discriminating whether a position of the projected pattern light is a light area or a dark area based on an output of the illuminance sensor at each time interval for each projected pattern light; a position specifying means for specifying a position of the illuminance sensor based on a result of the determination, the position specifying means includes an existence region narrowing means for narrowing down an existence region of each projection of the pattern light to either a bright region or a dark region based on a result of the discrimination, the sequential projection means repeats the projection of the pattern light onto the narrowed-down existence region each time the existence region is narrowed down based on the result of the discrimination between the bright region and the dark region, The position specifying means specifies the position of the illuminance sensor based on a presence area when a predetermined termination condition is satisfied.

2. the sequential projection means repeats projection of pattern light that divides the existence region into equal light regions and dark regions; 2. The projection image control system according to claim 1, wherein the existence area narrowing means narrows down the existence area of ​​the illuminance sensor for each projection of the pattern light.

3. the sequential projection means repeats projection of a pattern light that divides the existence region into equal bright and dark regions in a horizontal direction; 3. The projection image control system according to claim 2, wherein the existence area narrowing means narrows down the existence area in the horizontal direction of the illuminance sensor for each projection of the pattern light.

4. the sequential projection means repeats projection of a pattern light that divides the existence region into equal bright and dark regions in a vertical direction; 3. The projection image control system according to claim 2, wherein the existence area narrowing means narrows down the existence area in the vertical direction of the illuminance sensor for each projection of the pattern light.

5. the sequential projection means projects a pair of pattern lights in which the arrangements of the bright and dark regions are reversed to each other; The projection image control system according to claim 1, characterized in that the existence area narrowing means narrows down the existence area of ​​an illuminance sensor that is determined to be a bright area when one of a pair of pattern lights is projected and that is determined to be a dark area when the other of the pair of pattern lights is projected, to the bright area when the one of the pattern lights is projected.

6. 6. The projection image control system according to claim 5, wherein the position identifying means identifies the position of an illuminance sensor that is determined to be a bright area for both one and the other of the pair of pattern lights on the boundary between the areas.

7. the sequential projection means alternately projects a pattern light that divides the existence region into two equal parts, a bright region and a dark region, in the horizontal direction and a pattern light that divides the existence region into two equal parts, a bright region and a dark region, in the vertical direction; 2. The projection image control system according to claim 1, wherein the existence area narrowing means repeatedly narrows down the existence area of ​​the illuminance sensor alternately in each of the horizontal and vertical directions.

8. the sequential projection means projects a pair of pattern lights in which the arrangements of the bright and dark regions are reversed to each other; The projection image control system according to claim 1, characterized in that the existence area narrowing means narrows down the existence area of ​​an illuminance sensor that is determined to be a bright area when one of a pair of pattern lights is projected to the bright area of ​​that pattern light, and narrows down the existence area of ​​an illuminance sensor that is determined to be a bright area when the other of the pair of pattern lights is projected to the bright area of ​​that pattern light.

9. A projection image control system in which a projection object is provided with a plurality of illuminance sensors whose relative positions with respect to a projection target area are known, and projection onto the projection target area is controlled based on the position of each illuminance sensor in a pixel coordinate system, a sequential projection means for sequentially projecting a plurality of pattern lights, each having a bright region and a dark region arranged in a pixel coordinate system, onto an area where the illuminance sensor is present while switching the pattern lights at a predetermined time interval; a light / dark discrimination means for discriminating whether a position of the projected pattern light is a light area or a dark area based on an output of the illuminance sensor at each time interval for each projected pattern light; a position specifying means for specifying a position of the illuminance sensor based on a result of the determination, the light and dark regions of each of the pattern lights are arranged so that a time series of light and dark at a projection destination where the pattern lights are sequentially projected is unique for each position; the light / dark discrimination means includes a light / dark time series recording means for recording whether a position of each of the sequentially projected pattern lights is a light region or a dark region based on an output from an illuminance sensor, the position specifying means includes means for narrowing down an area in which the illuminance sensor is present based on the time series record; A projection image control system characterized in that the presence area is further narrowed down based on the time series of light and dark that is recorded by irradiating each pattern light onto the narrowed down presence area, and the presence area is repeatedly narrowed down.

10. the sequential projection means switches the projected pattern light at a predetermined time period; The projection image control system according to any one of claims 1 to 9, characterized in that the light / dark discrimination means discriminates an area where an illuminance sensor detects an illuminance value exceeding a predetermined threshold within the projection time of each pattern light as a bright area of ​​that pattern light.

11. The projection image control system according to claim 10, characterized in that the light / dark discrimination means sets the predetermined threshold value to an intermediate value between the illuminance value when a pattern light is projected such that the entire existence area is a light area and the illuminance value when a pattern light is projected such that the entire existence area is a dark area.

12. 10. The projection image control system according to claim 1, further comprising an optical mask for blocking incident light to each of the illuminance sensors with a predetermined mask pattern.

13. A projection image control method in which a projection object is provided with a plurality of illuminance sensors whose relative positions with respect to a projection target area are known, and a computer controls projection onto the projection target area based on the positions of the illuminance sensors in a pixel coordinate system, comprising: a step of sequentially projecting a plurality of pattern lights, each having a bright region and a dark region arranged in a pixel coordinate system, onto an area where an illuminance sensor is present while switching the pattern lights at a predetermined time interval; determining whether the position of each projected pattern light is a bright area or a dark area based on an output of the illuminance sensor at each time interval; determining a position of the illuminance sensor based on a result of the determination; The step of identifying the position includes narrowing down an area where the pattern light is present to either a bright area or a dark area based on a result of the determination for each projection of the pattern light; the step of sequentially projecting includes repeating the projection of the pattern light onto the narrowed-down existence region each time the existence region is narrowed down based on the result of the discrimination between the bright region and the dark region; The projection image control method, wherein the step of specifying the position specifies the position of the illuminance sensor based on a presence area when a predetermined termination condition is satisfied.

14. A projection image control program is provided for providing a projection object with a plurality of illuminance sensors, the relative positions of which with respect to a projection object area are known, and controlling projection onto the projection object area based on the positions of the illuminance sensors in a pixel coordinate system, the program comprising: a step of sequentially projecting a plurality of pattern lights, each having a bright region and a dark region arranged in a pixel coordinate system, onto an area where an illuminance sensor is present while switching the pattern lights at a predetermined time interval; a step of determining whether the position of each projected pattern light is a bright area or a dark area based on an output of the illuminance sensor at each time interval; specifying a position of the illuminance sensor based on a result of the determination; The step of identifying the position includes narrowing down an area where the pattern light is present to either a bright area or a dark area based on a result of the discrimination for each projection of the pattern light; the step of sequentially projecting includes repeating the projection of the pattern light onto the narrowed-down existence region each time the existence region is narrowed down based on the result of the discrimination between the bright region and the dark region; The step of specifying the position of the projection image control program is characterized in that the step of specifying the position of the illuminance sensor based on a presence area when a predetermined termination condition is satisfied.

Images