Portable projection device

Abstract

Provided is a portable projection device with which it is possible to easily correct camera shake without previously setting a projected position even in a situation where the camera shake occurs frequently.　A feature point extraction unit (20) extracts a first feature point from a first captured image and extracts a second feature point from a second captured image. A relative position change amount acquisition unit (21) detects the first feature point and the second feature point corresponding to each other on the basis of the comparison between the feature amount of the first feature point and the feature amount of the second feature point, and acquires the amount of change in the relative positional relationship between the first and second captured images from the positional relationship between the first feature point and the second feature point corresponding to each other. A projection image control unit (22) controls the projection direction and the like of the projection image in accordance with the amount of change in the relative positional relationship.

Description

Portable projection device

[0001] The present invention relates to a portable projection device.

[0002] In Patent Document 1, a light beam is projected onto a screen to project and display a projected image, and a deviation of the projected image with respect to the screen is detected. An additional optical system provided between the device that generates the projected image and the screen corrects the deviation. In Patent Document 1, marker groups on the screen side are arranged at the four corners of the screen, and marker groups on the projected image side are depicted at the four corners in the projected image. Then, after photographing the marker group on the screen side and the marker group on the projected image side and capturing them as images, from the video information of the marker group on the screen side and the marker group on the projected image side, parameters of three elements including the positional deviation, size deviation, and twist angle between the projected image and the screen are extracted. The additional optical system having three optical systems formed corresponding to these three-element parameters optically corrects each deviation by moving or deforming the additional optical system based on the extracted three-element parameters. The series of operations from the depiction of the above marker groups to the optical correction of the deviation are performed simultaneously as a feedback loop and sequentially and automatically, gradually adjusting the projected image to the screen.

[0003] Japanese Patent Laid-Open No. 10-333088

[0004] The projection-type image display device of Patent Document 1 requires a dedicated screen for displaying the projected image, and is mainly used as a stationary type placed on a table or the like. On the other hand, in recent years, a handy-type projection device that is held by hand and projects a projected image onto an arbitrary projection location such as a wall is being used. In the case of such a handy-type projection device, since it is in a situation where camera shake frequently occurs, it has been required to be able to easily perform camera shake correction without presetting the projection position on the screen.

[0005] An object of the present invention is to provide a portable projection device that can easily perform camera shake correction without presetting the projection position even in a situation where camera shake frequently occurs.

[0006] The portable projection device of the present invention comprises an image projection unit that projects a projection image, an imaging unit that captures at least a portion of the projection surface on which the projection image is projected as an imaging area and acquires a first imaging image and a second imaging image with different imaging timings, and a processor. The processor extracts a first feature point from the first imaging image and a second feature point from the second imaging image, detects corresponding first and second feature points based on a comparison of the feature quantities of the first and second feature points, obtains the amount of change in the relative positional relationship between the first and second imaging images from the positional relationship between the corresponding first and second feature points, and controls the projection direction or position of the projection image according to the amount of change in the relative positional relationship.

[0007] Preferably, the imaging area is the area outside the projection image region of the projection plane, the first feature point is extracted from the area outside the projection image region of the first captured image, and the second feature point is extracted from the area outside the projection image region of the second captured image. Preferably, the imaging area is the projection image overlap region of the projection plane that overlaps with the projection image region, the first feature point is extracted from the projection image overlap region of the first captured image, and the second feature point is extracted from the projection image overlap region of the second captured image. Preferably, there are multiple first feature points or second feature points. Preferably, there are multiple first feature points and second feature points.

[0008] Preferably, the processor extracts a first feature point and a second feature point each time a first and second image are acquired, and obtains the amount of change in their relative positional relationship. Preferably, the processor restricts the extraction of the first feature point until it becomes impossible to extract the first or second feature point, and uses the extracted first feature point and the second feature point sequentially extracted from the second image to obtain the amount of change in their relative positional relationship.

[0009] Preferably, if at least one of the first or second feature points becomes unextractable, the processor extracts the first and second feature points anew and uses the newly extracted first and second feature points to obtain the amount of change in their relative positional relationship. Preferably, if the first and second feature points become unextractable, it is because they are outside the imaging area, or because they are included within the area of ​​the projected image, and as a result, the feature quantity of the first or second feature point changes due to the projected image.

[0010] Preferably, the image projection unit has a sensor that detects changes in its position or orientation, and the acquisition of the amount of change in relative positional relationship is performed when the sensor detects a change in position or orientation that exceeds a threshold. The projection surface is provided with position detection markers, and preferably the processor acquires the amount of change in relative positional relationship from the positional relationship between the first feature point, the second feature point, the position detection marker included in the first captured image, and the position detection marker included in the second captured image.

[0011] The processor preferably corrects the projection direction so as to eliminate positional shifts based on changes in relative positional relationships. If the area of ​​the projected image is narrower than the projectable area, the processor preferably corrects the position of the projected image within the projectable area so as to eliminate positional shifts based on changes in relative positional relationships.

[0012] According to the present invention, even in situations where camera shake frequently occurs, image stabilization can be easily performed without pre-setting the projection position on the screen.

[0013] This is a schematic diagram of a portable projection device. This is a block diagram showing the functions of a portable projection device. (A) is an explanatory diagram showing the shooting area outside the projection image area, and (B) is an explanatory diagram showing the shooting area of ​​the overlapping projection image area. This is an explanatory diagram showing the imaging timing of the first and second captured images. This is a block diagram showing the functions of the projection control processor device. This is an explanatory diagram showing the first and second feature points. This is an explanatory diagram showing the amount of change in the relative positional relationship between each of the first and second feature points. This is an explanatory diagram showing how to change the projection direction. This is an explanatory diagram showing how to correct the position of the projected image. (A) is an explanatory diagram showing the case where the imaging timing of the first and second captured images and the acquisition timing of the change in the relative positional relationship are almost the same, and (B) is an explanatory diagram showing the case where the imaging timing of the first and second captured images and the acquisition timing of the change in the relative positional relationship are not the same, and the change in the relative positional relationship is acquired at a certain interval. (A) is an explanatory diagram showing a set of first and second captured images P1 and P2 assembled in a different way from (B) and (A). (B) is an explanatory diagram showing a set of first and second captured images P1 and P2 assembled in a different way from (A). (A) is an explanatory diagram showing a method for obtaining the amount of change in relative positional relationship until feature point extraction becomes impossible. (A) shows a method for obtaining the amount of change in relative positional relationship after feature point extraction becomes impossible, which is different from (B), and (B) is an explanatory diagram showing a method for obtaining the amount of change in relative positional relationship after feature point extraction becomes impossible, which is different from (A). This is a block diagram showing the functions of a portable projection device when a sensor is provided. This is a schematic diagram showing a screen and a position detection marker.

[0014] As shown in Figure 1, the portable projection device 10 projects an image toward the screen SC. The portable projection device 10 is equipped with a grip GP that the user holds in their hand, allowing the user to freely change the projection direction of the image. As a result, the portable projection device 10 can project images not only toward a regular projector screen, but also toward a wall or other surface.

[0015] As shown in Figure 2, the portable projection device 10 comprises an image projection unit 11, a shooting unit 12, and a projection control processor device 13. The image projection unit 11 projects a projection image. The projection image is projected onto a screen SC (see Figures 1 and 3). The shooting unit 12 is a digital camera or the like, and it captures at least a portion of the projection surface (screen SC) onto which the projection image is projected as a shooting area, thereby acquiring a first captured image and a second captured image with different capture timings.

[0016] The positional relationship between the image projection unit 11 and the imaging unit 12 is determined so that the imaging area and the projected image area are in the following positions. As shown in Figure 3(A), the imaging area IR is preferably the projection image area outside PRO, which is outside the projection image area PR on the projection surface (screen SC) onto which the projected image is projected, or as shown in Figure 3(B), the imaging area IR is preferably the projection image overlap area PRM, which overlaps with the projection image area PR on the projection surface. Regarding the overlap between the imaging area IR and the projection area PR, there are forms in which the projection area PR is included in the imaging area IR, forms in which the imaging area IR is included in the projection area PR, forms in which the imaging area IR and the projection area PR coincide, and forms in which the imaging area IR and the projection area PR partially overlap each other.

[0017] As shown in Figure 4, the imaging unit 12 acquires the first image P1 and the second image P2 alternately by imaging the imaging area at different imaging timings. These sets of first and second images are transmitted to the projection control processor device 13 and used for controlling the projected image. The positional relationship between the image projection unit 11 and the imaging unit 12 may be fixed or changeable. The imaging timing does not matter if the start and end timings of imaging are different, as long as the images are captured at the same moment (the same applies to the acquisition timing). Furthermore, since the imaging timing is assigned to each timing at which the first image P1 and the second image P2 are obtained, the imaging timing of the first image P1 and the imaging timing of the second image P2 are different.

[0018] As shown in Figure 5, the projection control processor device 13 includes a feature point extraction unit 20, a relative position change amount acquisition unit 21, and a projection image control unit 22. The portable projection device 10 is provided with a program memory (not shown) for executing various processes. The processor provided in the portable projection device 10 executes the program in the program memory, thereby realizing the functions of the feature point extraction unit 20, the relative position change amount acquisition unit 21, and the projection image control unit 22.

[0019] The feature point extraction unit 20 extracts first feature points from the first captured image P1 and second feature points from the second captured image P2. Specifically, as shown in Figure 6, the feature point extraction unit 20 extracts three first feature points C1a, C1b, and C1c from the first captured image P1 and three second feature points C2a, C2b, and C2c from the second captured image P2. The first and second feature points C1a, C2a, C1b, C2b, C1c, and C2c may be feature points that originally exist on the projection surface, such as scratches or indentations, or they may be feature points that are artificially created when using the projection surface. Here, in order to clearly represent the differences in feature quantities, they are represented by their respective shapes (circle, triangle, square). Furthermore, in order to accurately obtain the amount of change in the relative positional relationship between the first and second captured images, as described later, it is preferable to have multiple first feature points or second feature points, or to have multiple first and second feature points. When extracting multiple feature points, it is preferable that each feature point is spaced apart from the others. For example, feature points may be extracted from a point that straddles the center of the projection area PR, or feature points may be extracted from feature points that are provided at the edges of the imaging area IR.

[0020] Furthermore, if the imaging area IR is outside the projection image area PRO (see Figure 3(A)), the first feature point is extracted from the outside the projection image area PRO in the first captured image P1, and the second feature point is extracted from the outside the projection image overlapping area PRO in the second captured image. This prevents the projection image from acting as noise during the extraction of the first and second feature points. Also, if the imaging area IR is the projection image overlapping area PRM (see Figure 3(B)), the first feature point is extracted from the projection image overlapping area PRM in the first captured image P1, and the second feature point is extracted from the projection image overlapping area PRM in the second captured image. This prevents the first and second feature points from becoming unextractable even when the projection plane (screen SC) is finite.

[0021] The relative position change acquisition unit 21 detects corresponding first and second feature points based on a comparison of the feature quantities of the first feature point and the second feature point, and acquires the amount of change in the relative positional relationship between the first and second captured images from the positional relationship between the corresponding first and second feature points. Specifically, as shown in Figure 7, the relative position change acquisition unit 21 compares the feature quantities of the first feature points C1a, C1b, and C1c with the feature quantities of the second feature points C2a, C2b, and C2c, and detects corresponding first and second feature points. In this embodiment, first and second feature points with the same feature quantities are detected as corresponding first and second feature points. Therefore, the second feature point corresponding to the first feature point C1a is designated as the second feature point C2a, which has the same feature quantity as the first feature point C1a, being a "circle". Furthermore, the second feature point corresponding to the first feature point C1b is defined as the second feature point C2b, which has the same feature quantity as the first feature point C1b, with a "triangular" shape. Similarly, the second feature point corresponding to the first feature point C1c is defined as the second feature point C2c, which has the same feature quantity as the first feature point C1c, with a "square" shape. Note that for corresponding first and second feature points, this includes cases where the feature quantities of the first and second feature points are completely identical, as well as cases where the difference in feature quantities between the first and second feature points is within an acceptable range, such as an error.

[0022] The relative position change amount acquisition unit 21 then calculates the position change amount Da from the positional relationship between the corresponding first feature point C1a and the second feature point C2a. Similarly, it calculates the position change amount Db from the positional relationship between the corresponding first feature point C1b and the second feature point C2b, and calculates the position change amount Dc from the positional relationship between the corresponding first feature point C1c and the second feature point C2c. Based on the position change amounts Da, Db, and Dc, the relative position change amount acquisition unit 21 acquires the change amount D of the relative positional relationship between the first captured image P1 and the second captured image P2. Preferably, the change amount D of the relative positional relationship is a representative value (average value, maximum value, etc.) of the position change amounts Da, Db, and Dc.

[0023] Here, the relative position change D occurs not only when the portable projection device 10 moves in a plane parallel to the projection surface, when it moves in a direction having a component of the normal vector direction N (see Figure 1) of the projection surface, when the angle between the projection direction PD (see Figure 1) of the portable projection device 10 and the normal vector direction N of the projection surface changes, and as a result of a combination of these effects. Therefore, the method for calculating the relative positional relationship change D based on the positional change amounts Da, Db, and Dc differs depending on the cause of the relative position change.

[0024] The projection image control unit 22 controls the projection direction of the image projection unit 11 or the position of the projected image according to the amount of change D in the relative positional relationship. Specifically, as shown in Figure 8, the projection direction of the image projection unit 11 is changed from projection direction PD1 to projection direction PD2 so as to eliminate positional misalignment based on the amount of change D in the relative positional relationship. In this case, trapezoidal correction is performed according to the projection angle (trapezoidal correction based on the angle between projection direction PD1 and projection direction PD2). Alternatively, as shown in Figure 9, if the projection image area PR is narrower than the projectable area PRX, the position of the projected image is corrected within the projectable area PRX so as to eliminate positional misalignment based on the amount of change D in the relative positional relationship. As described above, in this embodiment, feature points on the screen SC that can serve as landmarks for position detection, such as irregularities on a wall, are extracted not only from the first captured image P1 but also from the second captured image P2, which is captured at a different timing than the first captured image P1. By using the amount of positional change of these extracted feature points, it is possible to control the position of the projected image (camera shake correction) without pre-setting markers or the like on the screen SC.

[0025] In the projection control processor device 13, the timing for acquiring the amount of change in relative positional relationship is controlled according to predetermined conditions. Each time a first image P1 and a second image P2 are acquired, the feature point extraction unit 20 extracts the first feature point C1 and the second feature point C2, and the relative position change amount acquisition unit 21 acquires the amount of change D in the relative positional relationship. In this case, each time a first image P1 and a second image P2 are acquired, the first feature point C1 and the second feature point C2 are extracted again.

[0026] Specifically, as shown in Figure 10(A), the change in relative position D is continuously acquired, roughly coinciding with the imaging timing of the imaging unit 12. Alternatively, as shown in Figure 10(B), the change in relative position D may be acquired at regular intervals, without coinciding with the imaging timing of the imaging unit 12. In the case of Figure 10(A), image stabilization can be performed with high accuracy, while in the case of Figure 10(B), image stabilization can be performed with a certain level of accuracy while saving power and improving battery life.

[0027] On the other hand, the feature point extraction unit 20 may restrict (stop) the extraction of the first feature point C1 until at least one of the first feature point C1 or the second feature point C2 becomes unextractable. The relative position change amount acquisition unit 21 uses the first feature point C1 already extracted by the feature point extraction unit 20 and the second feature point C2 sequentially extracted from the second image P2 to acquire the relative position change amount D. If at least one of the first feature point C1 or the second feature point C2 becomes unextractable by the feature point extraction unit 20, the first feature point C1 and the second feature point C2 are newly extracted, and the relative position change amount acquisition unit 21 uses the newly extracted first feature point C1 and the second feature point C2 to acquire the relative position change amount D.

[0028] Furthermore, the combination of sets consisting of the first captured image P1 and the second captured image P2 for extracting the first and second feature points C1 and C2 does not have to be unique. For example, as shown in Figure 11(A), sets may be defined in order of imaging timing, such as set S1 of (first captured image P1, second captured image P2), set S2 of (first captured image P1, second captured image P2) at the next imaging timing after set S1, ..., and the first and second feature points C1 and C2 may be extracted for each set S1, S2. Alternatively, as shown in Figure 11(B), when sets are defined for each imaging timing, such as set S1 of (first captured image P1, second captured image P2), set S1a of (second captured image P2 (reusing the second captured image P2 from set S1), first captured image P1) at the next imaging timing after set S1, ..., if the captured image from the previous imaging timing is reused to define the set, the feature points extracted in the previous set can also be reused. In this case, the burden of extracting feature points can be reduced.

[0029] For example, as shown in Figure 12, if the first feature point C1 and the second feature point C2 are extracted from the first image P1 and the second image P2 at acquisition timing T1, the first feature point C1 extracted from the first image P1 at acquisition timing T1 will not be updated until it becomes impossible to extract feature points from the first image P1 and the second image P2 at acquisition timing T4. Therefore, at acquisition timings T2 and T3, the first feature point C1 is not extracted from the first image P1, and the second feature point C2 is extracted from the second image P2 alone. This allows for image stabilization with a certain level of accuracy while saving power and improving battery life.

[0030] If at least one of the first or second feature points becomes unextractable, it may be because the first or second feature point is outside the imaging area IR, or because it is included in the projection image area PR, causing the feature quantity of the first or second feature point to change due to the projection image (see Figures 3(A) and 3(B)). The acquisition timing is assigned as the timing for acquiring the first and second captured images P1 and P2, which are a set of consecutive imaging timings.

[0031] On the other hand, as shown in Figure 13(A), if it becomes impossible to extract the second feature point C2 in the second image P2 acquired at acquisition timing T4, feature point extraction is performed in the first image P1 and the second image P2 acquired at acquisition timing TK (K and L are natural numbers of 5 or more) after acquisition timing T4 until feature point extraction becomes possible. In this case, it is preferable to move the portable projection device 10 to move the projection area PR to an area where feature points can be easily extracted, or to change to a screen SC that facilitates feature point extraction.

[0032] Furthermore, even if extraction of the second feature point C2 becomes impossible in the second image P2 at acquisition timing T4, it is not necessary to immediately resume extraction of the first feature point C1 and the second feature point C2. In this case, as shown in Figure 13(B), extraction of the first feature point C1 may be temporarily resumed from the first image P1 at acquisition timing T5, and if extraction of the first feature point C1 also becomes impossible, feature point extraction may be performed in the first image P1 and the second image P2 until feature point extraction becomes possible (K and L are natural numbers greater than or equal to 6). Therefore, if extraction of the first feature point C1, which was temporarily resumed, becomes possible, the extraction of the first feature point C1 and the second feature point C2 is not resumed, and the system returns to extracting only the second feature point C2.

[0033] Then, when the first feature point C1 and the second feature point C2 are extracted from the first captured image P1 and the second captured image P2 at acquisition timing TL (L is a natural number greater than or equal to K), the relative position change amount acquisition unit 21 uses the newly extracted first feature point C1 and the second feature point C2 to acquire the amount of change D in the relative positional relationship.

[0034] The timing for acquiring the amount of change in relative position may be controlled using a sensor 30 provided on the image projection unit 11, as shown in Figure 14. This allows for improved battery life through power saving while maintaining a certain level of accuracy in image stabilization. The sensor 30 detects changes in the position or orientation of the image projection unit 11. The relative position change acquisition unit 21 acquires the amount of change D in relative position when the sensor 30 detects a change in position or orientation that exceeds a threshold. Therefore, if the change in position or orientation detected by the sensor 30 is less than the threshold, the already acquired amount of change D in relative position continues to be used, and when the change in position or orientation exceeds the threshold, a new amount of change D in relative position is acquired and updated.

[0035] Furthermore, when acquiring a new change in relative positional relationship D, in addition to extracting the first feature point C1 and the second feature point C2 respectively and acquiring the change in relative positional relationship D (see Figures 10(A) and (B)), if the first feature point C1 is extracted, the change in relative positional relationship D may be acquired using the extracted first feature point C1 and the second feature point C2 which is sequentially extracted from the second image P2, until at least one of the first feature point C1 or the second feature point C2 becomes unextractable (see Figure 12).

[0036] In the above embodiment, the position of the projected image is controlled using first and second feature points extracted from the first and second captured images, but the position of the projected image may also be controlled by combining other information with the first and second feature points. For example, as shown in Figure 15, position detection markers 40 are provided on the screen SC, which is the projection surface. The position detection markers 40 are provided within the imaging area IR on the screen SC. The relative position change amount acquisition unit 21 then acquires the relative positional change amount D from the first feature points, the second feature points, the position detection markers 40 included in the first captured image, and the position detection markers 40 included in the second captured image. After acquiring the relative positional change amount D, the position of the projected image is controlled in the same manner as in the above embodiment. It is preferable that there are multiple position detection markers 40 (four at the four corners of the screen SC in Figure 15), and that the extraction accuracy of the feature point extraction unit 20 is above a certain level.

[0037] In this case, if the first and second feature points can be extracted, the change in relative position D is obtained from the positional relationship between the first and second feature points, which have the same feature quantities, as well as from the positional relationship between the position detection marker 40 included in the first captured image P and the position detection marker 40 included in the second captured image P2, similar to the embodiment described above. On the other hand, if the first and second feature points become unavailable, the change in relative position D is obtained only from the positional relationship between the position detection marker 40 included in the first captured image P and the position detection marker 40 included in the second captured image P2. Therefore, even if the first and second feature points become unavailable, it is not necessary to change the screen SC, and projection can continue without moving the position of the portable projection device 10, etc.

[0038] In this embodiment, each process is performed on any computer. Furthermore, any computer may perform these processes using a processor, a program, or a combination thereof. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other hardware element capable of running a program.

[0039] The processor may consist of one or more hardware components, and the type of hardware is not limited. For example, the processor may consist of programmable logic devices such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field Programmable Gate Array), dedicated circuits for executing specific processes such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). The processor also has various units or means that execute the various processes in this embodiment. Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, these components may reside in physically separate devices or in the same device. Furthermore, in any embodiment, the order of the processes performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware components are composed of electrical circuits (circuits) and the like, which are combinations of circuit elements such as semiconductor elements.

[0040] Furthermore, this embodiment may be implemented by hardware, software, firmware, microcode, or a combination thereof. The software, firmware, and microcode are composed of a program. The program may also be, for example, a group of program modules, each of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., recording media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on devices that are physically separated from each other. The program code or code segment may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. The program code or code segment may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0041] 10 Portable projection device 11 Image projection unit 12 Imaging unit 13 Projection control processor unit 20 Feature point extraction unit 21 Relative position change amount acquisition unit 22 Position correction unit 30 Sensor 40 Position detection marker GP Grip SC Screen PR Projection image area PRO Outside projection image area PRM Projection image overlap area IR Imaging area P1 First image capture P2 Second image capture C1 First feature point C2 Second feature point C1a, C1b, C1c First feature point C2a, C2b, C2c Second feature point Da, Db, Dc Position change amount D Change in relative positional relationship PD1, PD2 Projection direction PRX Projection range

Claims

1. A portable projection device comprising: an image projection unit that projects a projection image; an imaging unit that captures at least a portion of the projection surface onto which the projection image is projected as an imaging area and acquires a first imaging image and a second imaging image with different imaging timings; and a processor, wherein the processor extracts a first feature point from the first imaging image and a second feature point from the second imaging image; detects the first and second feature points corresponding to each other based on a comparison of the feature quantities of the first and second feature points; obtains the amount of change in the relative positional relationship between the first and second imaging images from the positional relationship between the corresponding first and second feature points; and controls the projection direction of the projection image or the position of the projection image according to the amount of change in the relative positional relationship.

2. The portable projection device according to claim 1, wherein the imaging area is outside the projection image area of ​​the projection surface, the first feature point is extracted from the first captured image outside the projection image area, and the second feature point is extracted from the second captured image outside the projection image area.

3. The portable projection device according to claim 1, wherein the imaging area is a projection image overlapping area of ​​the projection surface that overlaps with the area of ​​the projected image, the first feature point is extracted from the projection image overlapping area of ​​the first captured image, and the second feature point is extracted from the projection image overlapping area of ​​the second captured image.

4. The portable projection device according to claim 1, wherein there are multiple first feature points or second feature points.

5. The portable projection device according to claim 1, wherein there are multiple first and second feature points.

6. The portable projection device according to claim 1, wherein the processor extracts the first feature point and the second feature point each time the first image and the second image are acquired, and acquires the amount of change in the relative positional relationship.

7. The portable projection device according to claim 1, wherein the processor restricts the extraction of the first feature point until the first or second feature point becomes unextractable, and uses the extracted first feature point and the second feature point sequentially extracted from the second captured image to acquire the amount of change in the relative positional relationship.

8. The portable projection device according to claim 1, wherein if at least one of the first feature point or the second feature point becomes unextractable, the processor extracts the first and second feature points anew, and uses the newly extracted first and second feature points to obtain the amount of change in the relative positional relationship.

9. The portable projection device according to claim 7 or 8, wherein the first feature point and the second feature point become unextractable if they are outside the imaging area or if they are included within the area of ​​the projected image, causing the feature quantity of the first feature point or the feature quantity of the second feature point to change due to the projected image.

10. The portable projection device according to claim 1, comprising a sensor for detecting changes in the position or orientation of the image projection unit, wherein the acquisition of the amount of change in the relative positional relationship is performed when the sensor detects a change in position or orientation that exceeds a threshold.

11. The portable projection device according to claim 1, wherein a position detection marker is provided on the projection surface, and the processor obtains the amount of change in the relative positional relationship from the positional relationship between the first feature point, the second feature point, the position detection marker included in the first captured image, and the position detection marker included in the second captured image.

12. The portable projection device according to claim 1, wherein the processor corrects the projection direction so that the positional shift based on the amount of change in the relative positional relationship is eliminated.

13. The portable projection device according to claim 1, wherein the processor corrects the position of the projected image within the projectable area so as to eliminate positional displacement based on the change in the relative positional relationship when the area of ​​the projected image is narrower than the projectable area.

Images