Projection device

Abstract

To provide a projection device which can project an image to a desired position even if the projection device is moved.SOLUTION: A projection device 100 is for projecting an image, and includes: a projection control unit 470 for controlling projection of an image onto a predetermined position; and a moving amount calculation unit 420 for acquiring the acceleration of the projection device 100 and calculating the moving amount. The projection control unit 470 continues projection of the image to the predetermined position on the basis of the moving amount calculated by the moving amount calculation unit 420.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a projection device that is portable and capable of projecting an image. [Background technology]

[0002] 2. Description of the Related Art Miniaturized projection devices such as projectors have been developed. Some of the miniaturized projection devices are so-called wearable devices that are worn on the body of a user.

[0003] For example, JP2022-537574A (Patent Document 1) discloses a device including a configuration for projecting an image onto a part of a user's body and detecting the user's gestures with respect to the image. According to Patent Document 1, a wearable device is provided that can display an image at any location and function as a remote control for an air conditioner or the like.

[0004] However, in the conventional techniques including Patent Document 1, when the projection device moves while projecting an image, the projected image also moves. Therefore, the distance from the projection device to the projection surface changes, and the visibility of the image may decrease.

[0005] Therefore, there was a need for further technology that could continue to project an image at a specified position even if the projection device moved. Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the problems in the conventional technology described above, and has an object to provide a projection device that can continue to project an image at a desired location even when moved. [Means for solving the problem]

[0007] That is, according to the present invention, A projection device for projecting an image, comprising: A control means for controlling the projection of the image at a predetermined position; A calculation means for acquiring an acceleration of the projection device and calculating a movement amount; Including, The control means continues to project the image onto the predetermined position based on the amount of movement calculated by the calculation means. A projection device is provided. Effect of the Invention

[0008] According to the present invention, it is possible to provide a projection device that can continue to project an image at a desired location even when moved. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a projection device according to an embodiment of the present invention. [Diagram 2] 5A to 5C are diagrams showing an example of the operation of the projection device in the embodiment. [Diagram 3] FIG. 2 is a diagram showing a hardware configuration included in the projection device of the present embodiment. [Figure 4] FIG. 2 is a block diagram of software included in the projection device of the present embodiment. [Diagram 5] 6 is a flowchart showing a process in an initial projection mode according to the embodiment. [Figure 6] 5A and 5B are diagrams for explaining a movable range in the present embodiment. [Figure 7] 4 is a flowchart showing a process executed by the projection device of the present embodiment. [Figure 8] 5A and 5B are diagrams for explaining similar ranges of movement amounts in the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The present invention will be described below with reference to an embodiment, but the present invention is not limited to the embodiment described below. In addition, in each drawing referred to below, the same reference numerals are used for common elements, and the description thereof will be omitted as appropriate.

[0011] Hereinafter, an overview of this embodiment will be described with reference to Figs. 1 and 2. Fig. 1 is a diagram showing a schematic configuration of a projection device 100 in this embodiment. The projection device 100 of this embodiment shown in Fig. 1 is in a form that can be worn by a user as a so-called wearable device. The projection device 100 can be configured as a projector of various projection methods. The projection device 100 of this embodiment can be configured as, for example, a CRT projector, a liquid crystal projector, a DLP projector, an LCOS projector, a GLV projector, a laser projector, or the like, but this is not intended to limit the embodiment in particular.

[0012] The projection device 100 of this embodiment is worn by a user and can project an image at any desired location. For example, as shown in Fig. 1, the projection device 100 can project a projection image at a desired position on a projection surface. In the example of Fig. 1, a hand-shaped icon is projected onto the projection surface as a projection image. In this way, the user can display the projection image at any desired location, thereby projecting the image at, for example, a location to which the user wishes to draw attention.

[0013] Furthermore, the projection device 100 of this embodiment may include a camera that captures the projected image and the projection surface. The image captured by the camera may be transmitted to another device, for example, via a network. For example, as shown in FIG. 1, the transmitted image may be displayed on a display in a remote location, so that the projected image and the state of the projection surface may be communicated to a person in a remote location.

[0014] Next, FIG. 2 will be described. FIG. 2 is a diagram showing an example of the operation of the projection device 100 in this embodiment. In the case of the projection device 100 such as a wearable device as shown in FIG. 1, the user may move during image projection. For example, as shown in FIG. 2, consider a case where a user who is initially at position P0 moves to P1.

[0015] As shown in FIG. 2, when the user is at P0, the projection image is projected at position Q0 on the projection surface. If the user then moves to P1, the projection image will also move to position Q1 as the user moves. However, when the user wishes to project an image at position Q0 (for example, when the user wishes to draw attention to position Q0), it is not preferable for the projection image to move. Therefore, the projection device 100 of this embodiment is configured so that even when the user moves from P0 to P1, the projection image does not move and continues to be projected at position Q0.

[0016] Next, a description will be given of the hardware configuration of the projection device 100. Fig. 3 is a diagram showing the hardware configuration included in the projection device 100 of this embodiment. The projection device 100 includes a CPU 310, a RAM 320, a ROM 330, a storage device 340, a projection device 350, a camera 360, an acceleration sensor 370, and a communication I / F 380, and each piece of hardware is connected via a bus.

[0017] The CPU 310 is a device that executes a program that controls the operation of the projection device 100 and performs predetermined processing. The RAM 320 is a volatile storage device that provides an execution space for the program executed by the CPU 310, and is used for storing and extracting the program and data. The ROM 330 is a non-volatile storage device that stores the program executed by the CPU 310, firmware, and the like.

[0018] The storage device 340 is a readable / writable non-volatile storage device that stores an OS, various software, setting information, various data, and the like that operate the projection device 100. The storage device 340 of this embodiment can store, for example, a projected image, an image captured by the camera 360, and acceleration data values ​​acquired by the acceleration sensor 370. Examples of the storage device 340 include a hard disk drive (HDD) and a solid state drive (SSD).

[0019] The projection device 350 constitutes the projection means of this embodiment, and is an apparatus including a projection optical system according to various methods. The projection device 350 of this embodiment includes, for example, a light source, a lens, a focus adjustment mechanism, and the like.

[0020] The camera 360 constitutes the imaging means of this embodiment and is a device that captures an image of the surroundings of the projection device 100. The camera 360 of this embodiment can capture, for example, a projection image. In addition, the image captured by the camera 360 can be transmitted to another device via the communication I / F.

[0021] The acceleration sensor 370 constitutes the acceleration detection means of this embodiment, and is a device capable of detecting the acceleration at which the projection device 100 moves. The acceleration sensor 370 of this embodiment can detect acceleration components along three axes, namely the x-axis, y-axis, and z-axis.

[0022] The communication I / F 380 connects the projection device 100 to a network, enabling communication with other devices via the network. The communication via the network may be either wired communication or wireless communication, and various data can be transmitted and received using a predetermined communication protocol such as TCP / IP. The communication I / F 380 of this embodiment can transmit, for example, an image captured by the camera 360.

[0023] Note that the hardware components constituting the projection device 100 of this embodiment do not necessarily have to be included in one housing as shown in Figures 1 to 3. Therefore, for example, a configuration may be used in which the CPU 310 and the projection device 350 as a control device are included in one housing, and the camera 360 and the acceleration sensor 370 are included in one housing. Also, for example, a configuration may be used in which the CPU 310 and the camera 360 as a control device are included in one housing, and the projection device 350 and the acceleration sensor 370 are included in one housing.

[0024] The above describes the hardware configuration included in the projection device 100 of this embodiment. Next, the functional means executed by each piece of hardware in this embodiment will be described with reference to Fig. 4. Fig. 4 is a software block diagram included in the projection device 100 of this embodiment.

[0025] The projection device 100 includes the following functional units: an acceleration detection unit 410, a movement amount calculation unit 420, a movement amount similarity determination unit 430, an image capture unit 440, a captured image processing unit 450, a projection distance calculation unit 460, and a projection control unit 470. Each functional unit will be described below.

[0026] The acceleration detection unit 410 constitutes the acceleration detection means of this embodiment, and is means for detecting three-axis acceleration components output by the acceleration sensor 370. The acceleration detection unit 410 of this embodiment can output the detected acceleration data to the movement amount calculation unit 420. The acceleration detection unit 410 of this embodiment can detect acceleration according to the operating frequency of the acceleration sensor 370, and can detect acceleration at a frequency of, for example, about 500 Hz.

[0027] The movement amount calculation unit 420 constitutes the movement amount calculation means of this embodiment and is a means for calculating the movement amount. The movement amount calculation unit 420 of this embodiment can calculate the movement amount by converting the acceleration data detected by the acceleration detection unit 410 into the movement amount. Note that in other embodiments, the movement amount may be calculated based on data other than acceleration, for example, the movement amount may be calculated based on data detecting a change in position per unit time.

[0028] The movement amount similarity determination unit 430 constitutes the determination means of this embodiment, and is a means for making a determination regarding the movement amount calculated by the movement amount calculation unit 420. The movement amount similarity determination unit 430 of this embodiment can compare the calculated movement amount with the movement amount calculated immediately before the calculation, and determine whether the movement amounts are similar. The similarity of the movement amounts can be determined, for example, by comparing the vectors of the two movement amounts. In this way, by determining the similarity of the movement amount, even if the reliability of the acceleration sensor 370 is low, the movement amount of the projection device 100 can be appropriately handled, and the projection image can be projected at a predetermined position.

[0029] The photographing unit 440 is a means for controlling the operation of the camera 360 and photographing an image. The photographing unit 440 of this embodiment constitutes a photographing means. The image photographed by the photographing unit 440 is used, for example, to calculate the distance to the surface onto which the image is projected, or to calculate the amount of movement of the projection device 100. The photographed image may be transmitted to another device. The photographing unit 440 of this embodiment can photograph images at a frequency according to the frame rate of the camera 360, and can photograph images at a frequency of, for example, about 30 to 60 Hz.

[0030] The captured image processing unit 450 constitutes the image processing means of this embodiment, and is a means for performing various processes on the image captured by the imaging unit 440. The captured image processing unit 450 of this embodiment can calculate, for example, a feature amount of the image, and thereby can calculate the amount of movement of the projection device 100.

[0031] The projection distance calculation unit 460 constitutes the projection distance calculation means of this embodiment, and is a means for calculating the distance from the projection device 100 to the projection surface (hereinafter referred to as the projection distance) based on the image captured by the imaging unit 440. The projection distance calculated by the projection distance calculation unit 460 is output to the projection control unit 470.

[0032] The projection control unit 470 constitutes the projection control means of this embodiment, and is a means for projecting an image onto a projection surface by controlling the operation of the projection device 350. The projection control unit 470 of this embodiment can control the operation of the projection device 350 based on the projection distance calculated by the projection distance calculation unit 460, and can project an image onto a predetermined projection surface. In addition, when the projection device 100 moves, the projection control unit 470 can control the operation of the projection device 350 based on the amount of movement calculated from the captured image, acceleration data, and the like. As a result, as described in FIG. 2, an image can be projected at a fixed position even after the projection device 100 moves.

[0033] The above-mentioned software blocks correspond to functional means realized by causing each piece of hardware to function by executing the program of this embodiment by the CPU 310. Also, the functional means shown in each embodiment may be realized entirely by software, or part or all of them may be implemented as hardware that provides equivalent functions.

[0034] Furthermore, all of the above-mentioned functional means do not necessarily have to be included in the configuration shown in Fig. 4. For example, in another preferred embodiment, each functional means may be realized by the cooperation of multiple devices. Also, for example, a configuration like a projection system may be used in which a functional means corresponding to the image capturing unit 440 is disposed outside the projection device 100, and an image capturing the projection surface and the projected image is captured from the external image capturing unit 440.

[0035] Next, the process performed by the projection device 100 of this embodiment when it starts projection will be described with reference to Fig. 5. In the following, the process performed when starting a new image projection will be referred to as the "initial projection mode". Fig. 5 is a flowchart showing the process of the initial projection mode of this embodiment. The projection device 100 of this embodiment starts the process from step S1000.

[0036] In step S1000, the photographing unit 440 controls the operation of the camera 360 to photograph the location where the image is to be projected. Then, in step S1002, the projection distance calculation unit 460 calculates the distance to the projection surface based on the image photographed in step S1001.

[0037] Next, in step S1003, the projection control unit 470 calculates a projection angle based on the distance to the projection surface. After that, in step S1004, the projection control unit 470 controls the projection device 350 at the calculated projection angle to project an image.

[0038] After that, in step S1005, the projection device 100 ends the processing in the initial projection mode. Through the processing in the initial projection mode shown in Fig. 5, the projection device 100 can project an image at a predetermined position on the projection surface.

[0039] Incidentally, after starting to project an image by the process described in FIG. 5, if the projection device 100 moves, the projected image may move (see FIG. 2). In this case, it may be preferable to continue projecting the projected image at a fixed position from the viewpoint of drawing attention to the image. On the other hand, since the range in which the projection device 350 can project is limited by the performance of the projection optical system, if the projection device 100 moves a large amount, it may not be possible to project the image at a fixed position. In other words, the projection device 100 of this embodiment has a movable range for projecting an image at a fixed position.

[0040] Fig. 6 is a diagram for explaining the movable range in this embodiment. Fig. 6 shows an example in which a user wearing the projection device 100 in the form of a wearable device projects an image onto a projection surface. Here, as shown in Fig. 6, the projection device 100 is placed in a three-dimensional coordinate system consisting of an x-axis, a y-axis, and a z-axis.

[0041] The example of Fig. 6 shows the state of the projection device 100 and the projected image after the processing of the initial projection mode described in Fig. 5 is performed. As shown in Fig. 6, the user is at an initial position P0, and at this time, the projected image is projected at a position Q0.

[0042] The elliptical area indicated by the dashed line in FIG. 6 indicates the movable range of the projection device 100 in which the projection device 100 can continue to project the projection image onto Q0. The movable range is an area corresponding to the range in which an image can be projected onto a certain position, which is determined by the specifications and performance of various hardware constituting the projection device 100. The movable range of this embodiment may be determined, for example, from the projection angle of the projection device 350. In addition, when the projection angle is larger than the angle of view of the camera 360, the movable range may be determined from the angle of view. For example, when an image is projected with the vicinity of the edge of the area that the camera 360 can capture as the initial position, the projected image may not be included in the captured image after the movement depending on the direction in which the user moves. Therefore, in such a case, the movable range may be determined based on the angle of view. Furthermore, the extent to which the projected image is displaced from the predetermined position due to the movement of the projection device 100 is determined by the ratio of the depth before and after the movement. In other words, the movable range may also vary depending on the distance (depth) from the projection device 100 to the projection location. Therefore, the movable range may be determined by parameters such as the depth, the location to be projected, etc. Here, after the initial projection mode, if the projection device 100 moves outside the movable range, the projection image cannot be projected at the position Q0.

[0043] Therefore, the projection device 100 of this embodiment can project the projection image to the position Q0 as long as the movement is within the movable range. Hereinafter, the process executed by the projection device 100 of this embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart showing the process executed by the projection device 100 of this embodiment. Note that the process shown in FIG. 7 is repeated, for example, by the operation of the acceleration sensor 370. For example, when the operating frequency of the acceleration sensor 370 is 500 Hz, the process in FIG. 7 is performed 500 times per second.

[0044] Projection device 100 of this embodiment starts processing from step S2000. In step S2001, the processing branches depending on whether or not camera 360 has captured an image. Here, camera 360 captures images at a frame rate of about 30 to 60 Hz, which is sufficiently lower than the operating frequency of acceleration sensor 370. Therefore, while the processing of FIG. 7 is being performed, an image may or may not be captured. Therefore, in step S2001, the processing branches depending on whether or not an image has been captured.

[0045] In step S2001, if an image has been captured (YES), the process proceeds to step S2002. In step S2002, the image used in the subsequent processes is updated to the latest image data (i.e., the image data captured in step S2001). On the other hand, if an image has not been captured (NO), there is no image data to be updated, so the process proceeds to step S2003 without performing the process in step S2002.

[0046] Next, in step S2003, acceleration detection unit 410 acquires data on the acceleration at which projection device 100 moves. After that, in step S2004, movement amount calculation unit 420 calculates the movement amount of projection device 100 based on the acquired acceleration data. Note that the movement amount calculated here is treated as a predicted value at the time of step S2004.

[0047] Next, in step S2005, the process branches depending on whether the predicted movement amount is similar to the most recent movement amount. Similarity of the movement amount will now be described with reference to Fig. 8. Fig. 8 is a diagram for explaining a similarity range of the movement amount in this embodiment.

[0048] Fig. 8(a) shows an example of the movement of the projection device 100 from time t0 to t2. As shown in Fig. 8, the projection device 100 (and the user wearing the projection device 100) is at position P0 at time t0, at position P1 at time t1, and at position P2 at time t2. Here, it is assumed that the projection device 100 moves from position P0 to P1 by a movement amount v1 from time t0 to t1, and moves from position P1 to P2 by a movement amount v2 from time t1 to t2.

[0049] In this case, in the process of step S2005 in Fig. 7, the predicted movement amount (i.e., the latest movement amount) is v2, and the most recent movement amount is v1. Therefore, in step S2005, the movement amount similarity determination unit 430 determines whether v2 and v1 are similar.

[0050] Here, the similarity of the movement amount can be determined by the magnitude and direction of the movement amount. FIG. 8(b) shows an example of a similar range of the movement amount. The area shown in dark color in FIG. 8(b) shows a range of similar movement amounts with respect to the movement amount of v1 (the most recent movement amount). The movement amount similarity range may be defined by the magnitude and direction of a vector (hereinafter referred to as a reference vector) indicating the most recent movement amount. For example, a region in which the magnitude of a vector with respect to the reference vector is within a predetermined range and the direction of the vector with respect to the reference vector is within a predetermined range can be set as a range in which the movement amounts are similar. As shown in FIG. 8(b), the movement amount v2 is within the movement amount similarity range of the movement amount v1, so the movement amount similarity determination unit 430 can determine that v1 and v2 are similar.

[0051] In this way, by determining the similarity of the amount of movement, it is possible to appropriately correct the amount of movement for projecting the projection image at a fixed position. That is, for example, if the change in the amount of movement is extremely large and it is determined that the amount of movement is not similar, there is a possibility that the reliability of the acquired acceleration data is low due to some malfunction of the acceleration sensor 370. Therefore, by calculating the amount of movement not only from the amount of movement calculated from the acceleration but also from the image captured by the camera 360, it is possible to continue projecting the image at a specified position.

[0052] Returning to FIG. 7 for explanation, in step S2005, if the movement amounts are not similar (NO), the process proceeds to step S2006. In this case, since the reliability of the acceleration data is low, in step S2006, projection control unit 470 calculates a corrected movement amount based on the image data captured by camera 360 and the predicted movement amount value. In this case, if no image is captured in step S2001, the most recently captured image data (i.e., image data of the previous frame) is used to calculate the movement amount. In this way, by calculating the movement amount together with the captured image data, it is possible to calculate a more accurate movement amount even if the reliability of the acceleration data is low.

[0053] On the other hand, if the movement amounts are similar in step S2005 (YES), the process proceeds to step S2007. In step S2007, projection control unit 470 sets the predicted movement amount value as the movement amount data of projection device 100. In this way, if the movement amount is similar to the most recent movement amount, that is, if the reliability of the acceleration data is high, the calculation of the movement amount based on the captured image as in step S2006 is not performed, thereby reducing the processing load and allowing the process of projecting an image at a predetermined position to be continued in a relatively short processing time.

[0054] Then, in step S2008, the projection control unit 470 updates the depth information for projecting the image based on the amount of movement calculated or set in step S2006 or S2007. The projection control unit 470 of this embodiment updates the depth information up to the projection position after movement (the predetermined position to which the image is currently projected) based on, for example, the distance to the position to which the image is currently projected and the amount of movement.

[0055] Next, in step S2009, the process branches depending on whether or not the projector 100 after movement is within the movable range. Whether or not the projector 100 is within the movable range can be determined by the projection control unit 470 based on the depth information updated in step S2008, for example.

[0056] In step S2009, if it is not within the movable range (NO), the process proceeds to step S2011. In this case, since the projection device 100 has moved to a position where the projection device 350 cannot project an image at a predetermined position, the projection device 100 transitions to the initial projection mode in step S2011 and performs the series of processes shown in Fig. 5 to reset the position where the image is projected.

[0057] On the other hand, if it is within the movable range in step S2009 (YES), the process proceeds to step S2010. In step S2010, the projection control unit 470 projects the image. Here, the projection control unit 470 controls the operation of the projection device 350 so that the image is projected at the same position as the current projected position. The projection control unit 470 of this embodiment can continue to project the image at a predetermined position even after the projection device 100 is moved by generating an output signal of the image to be projected based on the updated depth information.

[0058] After projecting the image in step S2010, the process returns to step S2001 and repeats the above-described series of processes. This allows the projection device 100 of this embodiment to continue projecting the projected image at the same position without changing the position even if the projection device 100 moves.

[0059] The projection device 100 of this embodiment can continuously project the projection image at a position desired by the user by performing the process shown in Fig. 7. In addition, by selecting a calculation method of the movement amount that is the criterion for determining whether or not the movement amount is within the movable range depending on whether or not the movement amount is similar, it is possible to improve the reliability of the movement amount and speed up the process at the same time.

[0060] According to the embodiment of the present invention described above, it is possible to provide a projection device that can continue to project an image at a desired location even when moved.

[0061] Each function of the above-described embodiments of the present invention can be realized by a device-executable program written in C, C++, C#, Java (registered trademark), etc., and the program of this embodiment can be stored on a device-readable recording medium such as a hard disk drive, CD-ROM, MO, DVD, flexible disk, EEPROM (registered trademark), EPROM, etc. and distributed, and can also be transmitted over a network in a format that can be used by other devices.

[0062] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0063] Although the present invention has been described above with reference to an embodiment, the present invention is not limited to the above-described embodiment, and as long as the functions and effects of the present invention are achieved within the scope of embodiments that a person skilled in the art can imagine, they are included in the scope of the present invention. [Explanation of symbols]

[0064] 100...projection apparatus, 310...CPU, 320...RAM, 330...ROM, 340...storage device, 350...projection device, 360...camera, 370...acceleration sensor, 380...communication I / F, 410...acceleration detection unit, 420...movement amount calculation unit, 430...movement amount similarity determination unit, 440...photographing unit, 450...photographed image processing unit, 460...projection distance calculation unit, 470...projection control unit [Prior art documents] [Patent documents]

[0065] [Patent Document 1] Special Publication No. 2022-537574

Claims

1. A projection device for projecting an image, comprising: A control means for controlling the projection of the image at a predetermined position; A calculation means for acquiring an acceleration of the projection device and calculating a movement amount; Including, The control means continues to project the image onto the predetermined position based on the amount of movement calculated by the calculation means. Projection device.

2. The method further includes a determination unit that determines whether the movement amount calculated by the calculation unit is similar to a movement amount calculated immediately before.

2. The projection device of claim 1.

3. Further comprising an image capturing means for capturing the projected image, When the determination means determines that the similarity is not present, The control means projects an image onto the predetermined position based on the image captured by the imaging means and the amount of movement calculated by the calculation means.

3. The projection device of claim 2.

4. The frequency with which the calculation means acquires the acceleration is higher than the frequency with which the image capture means captures images.

4. The projection device according to claim 3.

5. When the image capturing means captures an image, data of the captured image used for calculating the amount of movement is updated.

5. The projection device according to claim 4.

6. The predetermined position onto which the image is projected is a projection position of the image at a time when the projection is started.

2. The projection device of claim 1.

7. the control means determines whether or not a movement amount from a position of the projection device at the time when the projection is started is within a movable range.

7. The projection device of claim 6.

8. the control means resets the projection position when the movement amount is not within the movable range.

8. The projection device of claim 7.

9. The movable range is determined based on a projection angle of the projection means and an angle of view of the imaging means.

8. The projection device of claim 7.

Images