Control device, moving body, control method, and control program

Abstract

Provided are a control device, a moving body, a control method, and a control program that make it possible to control the visibility of the position of a marker displayed on a projection surface together with a projection image.　A control device (14) instructs a projection device (11) to project a projection image (11b) onto a projection surface (6a). The control device (14) also acquires a picked-up image obtained by an image pickup device (12) picking up the projection surface (6a) and detects a specific indication marker (61) from the acquired picked-up image. The control device (14) then changes a partial region of the projection image (11b) based on the position of the indication marker (61).

Description

Control device, mobile object, control method, and control program

[0001] The present invention relates to a control device, a mobile object, a control method, and a control program.

[0002] Patent Document 1 describes a display method in which marker information indicating the characteristics of a marker is acquired, correspondence information that associates the image to be displayed with the marker information is generated, the position and characteristics of the marker placed on a screen are detected by a projector, a corresponding image is identified based on the marker information corresponding to the detected characteristics of the marker and the correspondence information, a display position of the image is determined based on the position of the detected marker, and the identified image is displayed at the determined display position.

[0003] Patent document 2 describes a projection device that controls switching between the operating state and the resting state of the projection means based on the shape of a mark read from the projection surface, and changes the display mode of the image displayed on the projection surface.

[0004] Patent document 3 describes a projector that projects a projection image including an object image onto a screen, detects the size of the area onto which the projection image is projected, detects operation of a pointer on the screen, and, if the detected operation of the pointer is an operation to move the object image, varies the amount of movement of the object image depending on the size of the image projection area.

[0005] Patent document 4 describes a projection projector in which a marker projection unit is provided in a remote control transmitter for projecting a marker onto a screen as a reference for adjusting the position of a projected image, the projector body is provided with a detection sensor that detects the marker projected onto the screen, and the projector body scales the image signal to be input to a display element so that the top or bottom edge of the projected image is aligned with the marker detected by the detection sensor, thereby moving the image projection position.

[0006] Japanese Patent Publication No. 2020-134895 Japanese Patent Publication No. 2019-004368 Japanese Patent Publication No. 2016-188892 Japanese Patent Publication No. 2005-039518

[0007] One embodiment of the technique of the present disclosure provides a control device, a moving body, a control method, and a control program that can control the visibility of the position of a marker that is displayed on a projection surface together with a projected image.

[0008] (1) A control device including a processor, wherein the processor instructs a projection device to project a first image onto a projection surface, acquires a second image obtained by an imaging device capturing an image of the projection surface, detects a specific marker from the second image, and modifies a partial area of ​​the first image based on the position of the marker.

[0009] (2) The control device according to (1), wherein the processor changes the partial area so as to reduce the visibility of the marker.

[0010] (3) The control device according to (1), in which the processor changes the partial area so as to increase the visibility of the marker.

[0011] (4) The control device according to any one of (1) to (3), wherein the processor displays a specific symbol image in the partial area of ​​the first image.

[0012] (5) The control device according to (4), wherein the processor changes the position of the symbol image in the first image in response to a change in the position of the marker.

[0013] (6) The control device according to (5), wherein the processor does not change the position of the symbol image in the first image in response to the change in the position of the marker when the change in the position of the marker satisfies a predetermined condition.

[0014] (7) The control device according to (6), wherein the processor changes the predetermined condition based on at least one of the size and brightness of the detected marker.

[0015] (8) The control device according to (6), wherein the processor changes the size of the symbol image in the first image based on at least one of the size and brightness of the detected marker.

[0016] (9) The control device according to any one of (5) to (8), wherein the processor displays, on the first image, an image representing a trajectory of a change in the position of the symbol image on the first image.

[0017] (10) The control device according to any one of (5) to (9), wherein the processor generates history data of changes in the symbol image in the first image.

[0018] (11) The control device according to any one of (5) to (10), wherein the processor limits a direction of change in the position of the symbol image in the first image.

[0019] (12) The control device according to any one of (5) to (11), wherein the processor changes the symbol image based on a change in the position of the marker in the first image.

[0020] (13) The control device according to any one of (5) to (12), wherein the processor holds the position of the symbol image when the marker that has been detected becomes undetectable.

[0021] (14) A control device according to any one of (5) to (13), wherein, when the position of the symbol image in the first image cannot be changed in accordance with a change in the position of the marker, the processor displays the symbol image at a position corresponding to the position of the marker within an area in the first image where the symbol image can be displayed.

[0022] (15) The control device according to any one of (4) to (14), wherein the processor changes the symbol image based on a state of the marker.

[0023] (16) The control device according to (15), wherein the state of the marker includes at least one of the color of the marker, the shape of the marker, the size of the marker, and a cycle of blinking of the marker.

[0024] (17) A control device according to any one of (4) to (16), wherein the processor displays the symbol image at a position different from the position of the marker in the first image depending on a difference between a first area on the projection surface onto which the first image is projected and a second area on the projection surface in which the marker can be detected, and when the marker is present in the first area, changes the part of the area so as to reduce the visibility of the marker.

[0025] (18) The control device according to any one of (1) to (17), wherein, when the marker that was detected becomes undetectable, the processor displays an image indicating that the marker cannot be detected on the first image.

[0026] (19) The control device according to any one of (1) to (18), wherein the marker is an invisible light ray projected onto the projection surface.

[0027] (20) The control device according to any one of (1) to (19), wherein the processor instructs the projection device to change at least one of the position and size of a projection area according to a detection result of the marker.

[0028] (21) The control device according to any one of (1) to (20), wherein the processor instructs the projection device to blink the first image.

[0029] (22) A moving body including the control device according to any one of (1) to (21), the projection device, and the imaging device, wherein the control device is capable of controlling the movement of the moving body.

[0030] (23) The moving body according to (22), wherein the processor performs the movement control to change at least one of the position and size of the projection area of ​​the projection device in accordance with a detection result of the marker.

[0031] (24) A control method, in which a processor instructs a projection device to project a first image onto a projection surface; an imaging device captures an image of the projection surface to obtain a second image; detects a specific marker from the second image; and modifies a partial area of ​​the first image based on the position of the marker.

[0032] (25) A control program for causing a processor to execute the following processes: instructing a projection device to project a first image onto a projection surface; acquiring a second image obtained by an imaging device capturing an image of the projection surface; detecting a specific marker from the second image; and modifying a partial area of ​​the first image based on the position of the marker.

[0033] According to the present invention, it is possible to provide a control device, a moving body, a control method, and a control program that are capable of controlling the visibility of the position of a marker that is displayed on a projection surface together with a projected image.

[0034] 1 is a diagram showing an example of a moving body 10 to which the control device of the present invention can be applied. FIG. 1 is a diagram showing an example of the configuration of the moving body 10. FIG. 2 is a schematic diagram showing an example of the internal configuration of the projection device 11. FIG. 3 is a diagram showing an example of the hardware configuration of an information terminal 40. FIG. 4 is a diagram showing an example of a projection area by the projection device 11 and an imaging range by the imaging device 12. FIG. 5 is a diagram showing an example of irradiation of an indication marker onto a projection surface 6a. FIG. 6 is a flowchart showing an example of processing by the control device 14. FIG. 7 is a diagram showing a first example of changing a partial area of ​​the projection image 11b based on the position of the indication marker 61. FIG. 8 is a diagram showing a second example of changing a partial area of ​​the projection image 11b based on the position of the indication marker 61. FIG. 9 is a diagram showing a third example of changing a partial area of ​​the projection image 11b based on the position of the indication marker 61. FIG. 10 is a diagram showing a fourth example of changing a partial area of ​​the projection image 11b based on the position of the indication marker 61. FIG. 11 is a diagram showing a fifth example of changing a partial area of ​​the projection image 11b based on the position of the indication marker 61. FIG. 12 is a diagram showing an example of movement of a symbol image 11d in response to movement of the indication marker 61. FIG. 13 is a diagram showing an example of non-following of the symbol image 11d with respect to hand movement of the indication marker 61, etc. FIG. 14 is a diagram showing an example of changing a predetermined condition based on the detected size of the indication marker 61, etc. 1 is a diagram showing an example of changing the size of the symbol image 11d based on the size etc. of the detected indication marker 61. FIG. 2 is a diagram showing an example of displaying a trajectory of movement of the symbol image 11d. FIG. 3 is a diagram showing an example of restricting the direction of change in the position of the symbol image 11d in the projection image 11b. FIG. 4 is a diagram showing an example of changing the symbol image 11d based on a change in the position of the indication marker 61. FIG. 5 is a diagram showing an example of maintaining the position of the symbol image 11d when the indication marker 61 can no longer be detected. FIG. 6 is a diagram showing an example of displaying an image indicating that the indication marker 61 cannot be detected. FIG. 7 is a diagram showing an example of displaying the symbol image 11d when the position of the symbol image 11d can no longer be changed. FIG. 8 is a diagram showing an example of displaying the symbol image 11d at a position different from the position of the indication marker 61. FIG. 9 is a flowchart showing another example of processing by the control device 14. FIG. 10 is a diagram showing an example of changing the position of the projection area 11a due to movement of the indication marker 61.

[0035] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0036] (Embodiment) <Mobile body 10 to which the control device of the present invention can be applied> Fig. 1 is a diagram showing an example of a mobile body 10 to which the control device of the present invention can be applied. As shown in Fig. 1, the mobile body 10 is a mobile flying object, such as an unmanned aerial vehicle also known as a drone. For example, the mobile body 10 is a multicopter having three or more rotors (for example, four rotors).

[0037] The mobile object 10 is equipped with a projection device 11 and an imaging device 12. The projection device 11 is a projection device capable of projecting onto a projection object 6. The imaging device 12 is capable of capturing an image of the projection object 6. The projection object 6 is an object such as a wall, and has a projection surface 6a onto which the projection device 11 projects. In the example shown in FIG. 1 , the projection object 6 is a rectangular parallelepiped.

[0038] The information terminal 40 is an information terminal carried by the user U. The information terminal 40 is capable of communicating with the mobile object 10. In the example of FIG. 1 , the information terminal 40 is a tablet terminal. However, the information terminal 40 is not limited to a tablet terminal, and may be various types of information terminals such as a smartphone, a notebook personal computer, or a desktop personal computer. The configuration of the information terminal 40 will be described with reference to FIG. 4 . The user U can perform various controls of the mobile object 10 by operating the information terminal 40.

[0039] <Configuration of Mobile Object 10> Fig. 2 is a diagram showing an example of the configuration of the mobile object 10. As shown in Fig. 2, the mobile object 10 includes, for example, a projection device 11, an imaging device 12, a control device 14, a communication unit 15, and a movement mechanism 16.

[0040] The projection device 11 is configured by, for example, a liquid crystal projector or a projector using LCOS (Liquid Crystal On Silicon), etc. In the following description, the projection device 11 will be described as a liquid crystal projector.

[0041] The imaging device 12 is an imaging unit including an imaging lens and an imaging element, such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor.

[0042] The control device 14 performs various controls on the mobile body 10. The control device 14 is an example of a control device of the present invention. The controls by the control device 14 include, for example, control of projection by the projection device 11, control of imaging by the imaging device 12, control of communication by the communication unit 15, and control of movement of the mobile body 10 by the movement mechanism 16.

[0043] The control device 14 is a device that includes a control unit composed of various processors, a communication interface (not shown) for communicating with each part of the mobile body 10, and a storage medium 14a such as a hard disk, SSD (Solid State Drive), or ROM (Read Only Memory), and it controls the mobile body 10 in an integrated manner. The various processors in the control unit of the control device 14 include a CPU (Central Processing Unit), which is a general-purpose processor that executes programs to perform various processes, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), or a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically for performing specific processes.

[0044] More specifically, the structure of these various processors is an electric circuit that combines circuit elements such as semiconductor devices. The control unit of the control device 14 may be configured with one of the various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0045] The communication unit 15 is a communication interface that allows communication with other devices, for example, a wireless communication interface that performs wireless communication with an information terminal 40 on the ground while the moving body 10 is flying.

[0046] The movement mechanism 16 is a mechanism for moving the moving body 10. For example, if the moving body 10 is a multicopter, the movement mechanism 16 includes four rotors, actuators such as motors that rotate the rotors, and a control circuit that controls the actuators. However, the number of rotors included in the movement mechanism 16 may be three, or five or more.

[0047] The projection device 11, the image capture device 12, the control device 14, the communication unit 15, and the movement mechanism 16 are realized, for example, as a single device mounted on the moving body 10. Alternatively, the projection device 11, the image capture device 12, the control device 14, the communication unit 15, and the movement mechanism 16 may be realized by a plurality of devices mounted on the moving body 10 and capable of cooperating by communicating with each other.

[0048] <Internal Configuration of Projection Device 11> Fig. 3 is a schematic diagram showing an example of the internal configuration of the projection device 11. As shown in Fig. 3, the projection device 11 of the moving body 10 shown in Fig. 2 includes a light source 31, a light modulation unit 32, a projection optical system 33, and a control circuit 34. The light source 31 includes a light-emitting element such as a laser or an LED (Light Emitting Diode), and emits, for example, white light.

[0049] The light modulation unit 32 is composed of three liquid crystal panels (light modulation elements) that modulate the color light emitted from the light source 31 and separated into three colors, red, blue, and green, by a color separation mechanism not shown, based on image information and emit each color image, and a dichroic prism that mixes the color images emitted from the three liquid crystal panels and emits them in the same direction. These three liquid crystal panels may be equipped with red, blue, and green filters, respectively, and the white light emitted from the light source 31 may be modulated by each liquid crystal panel to emit each color image.

[0050] The projection optical system 33 receives light from the light source 31 and the light modulation unit 32, and is configured by, for example, a relay optical system including at least one lens. The light that passes through the projection optical system 33 is projected onto the projection object 6.

[0051] The area of ​​the projection object 6 that is irradiated with light that passes through the entire range of the light modulation unit 32 is the projectable range in which projection can be performed by the projection device 11. Within this projectable range, the area that is irradiated with light that actually passes through the light modulation unit 32 is the projection area of ​​the projection device 11. For example, by controlling the size, position, and shape of the area of ​​the light modulation unit 32 through which light passes, the size, position, and shape of the projection area of ​​the projection device 11 can be changed within the projectable range.

[0052] The control circuit 34 controls the light source 31, the light modulation unit 32, and the projection optical system 33 based on the display data input from the control device 14, thereby projecting an image based on this display data onto the projection object 6. The display data input to the control circuit 34 is made up of three pieces of data: red display data, blue display data, and green display data.

[0053] Furthermore, the control circuit 34 changes the projection optical system 33 based on a command input from the control device 14, thereby enlarging or reducing the projection area of ​​the projection device 11. Furthermore, the control circuit 34 may change the projection optical system 33 based on a command input from the control device 14, thereby moving the projection area of ​​the projection device 11.

[0054] The projection device 11 also includes a shift mechanism that mechanically or optically shifts the projection area of ​​the projection device 11 while maintaining the image circle of the projection optical system 33. The image circle of the projection optical system 33 is an area through which projection light incident on the projection optical system 33 passes through the projection optical system 33 appropriately in terms of light intensity drop-off, color separation, peripheral curvature, etc.

[0055] The shift mechanism is realized by at least one of an optical system shift mechanism that performs an optical system shift and an electronic shift mechanism that performs an electronic shift.

[0056] The optical system shift mechanism is, for example, a mechanism that moves the projection optical system 33 in a direction perpendicular to the optical axis, or a mechanism that moves the light modulation unit 32 in a direction perpendicular to the optical axis instead of moving the projection optical system 33. The optical system shift mechanism may also be a mechanism that moves the projection optical system 33 and the light modulation unit 32 in combination.

[0057] The electronic shift mechanism is a mechanism that shifts the pseudo projection area by changing the range through which light is transmitted in the light modulation section 32 .

[0058] The projection device 11 may also include a projection direction change mechanism that moves the projection area together with the image circle of the projection optical system 33. The projection direction change mechanism is a mechanism that changes the projection direction of the projection device 11 by changing the orientation of the projection device 11 through mechanical rotation.

[0059] <Hardware configuration of information terminal 40> Fig. 4 is a diagram showing an example of the hardware configuration of the information terminal 40. The information terminal 40 shown in Fig. 1 includes, for example, a processor 41, a memory 42, a communication interface 43, and a user interface 44, as shown in Fig. 4. The processor 41, the memory 42, the communication interface 43, and the user interface 44 are connected by, for example, a bus 49.

[0060] The processor 41 is a circuit that performs signal processing, and is, for example, a CPU that controls the entire information terminal 40. The processor 41 may be realized by other digital circuits such as an FPGA or a DSP (Digital Signal Processor). The processor 41 may also be realized by combining multiple digital circuits.

[0061] The memory 42 includes, for example, a main memory and an auxiliary memory. The main memory is, for example, a RAM (Random Access Memory). The main memory is used as a work area for the processor 41.

[0062] The auxiliary memory is a non-volatile memory such as a magnetic disk, an optical disk, a flash memory, etc. The auxiliary memory stores various programs that operate the information terminal 40. The programs stored in the auxiliary memory are loaded into the main memory and executed by the processor 41.

[0063] The auxiliary memory may also include a portable memory that is removable from the information terminal 40. Portable memories include memory cards such as a USB (Universal Serial Bus) flash drive and an SD (Secure Digital) memory card, and external hard disk drives.

[0064] The communication interface 43 is a communication interface that performs wireless communication with an external device (for example, the communication unit 15 of the mobile object 10) of the information terminal 40. The communication interface 43 is controlled by the processor 41.

[0065] The user interface 44 includes, for example, an input device that accepts operation input from the user and an output device that outputs information to the user. The input device can be realized by, for example, keys (e.g., a keyboard) or a remote control. The output device can be realized by, for example, a display or a speaker. The input device and the output device may also be realized by, for example, a touch panel. The user interface 44 is controlled by the processor 41.

[0066] <Projection Area by Projection Device 11 and Imaging Range by Imaging Device 12> Fig. 5 is a diagram showing an example of the projection area by the projection device 11 and the imaging range by the imaging device 12. The projection area 11a shown in Fig. 5 is the projection area by the projection device 11. That is, a projection image 11b of the projection target by the projection device 11 is displayed in the projection area 11a. The projection image 11b may be a still image or a moving image.

[0067] The imaging range 12a is the imaging range of the imaging device 12. That is, imaging data of the imaging range 12a is obtained by the imaging device 12. In the example of Fig. 5, the imaging range 12a is wider than the projection area 11a, and the imaging range 12a includes the projection area 11a.

[0068] <Illumination of an indicator marker onto the projection surface 6a> Fig. 6 is a diagram showing an example of illumination of an indicator marker onto the projection surface 6a. For example, the user U carries a laser pointer 60, and can use the laser pointer 60 to illuminate an indicator marker 61 onto the projection surface 6a. The indicator marker 61 is, for example, made of visible light. Note that it is desirable that the projection image 11b projected onto the projection area 11a from the projection device 11 does not include an image similar to the indicator marker 61.

[0069] <Processing by Control Device 14> Fig. 7 is a flowchart showing an example of processing by the control device 14. The control device 14 executes, for example, the processing shown in Fig. 7. First, the control device 14 instructs the projection device 11 to start projecting the projection image 11b (step S11). This causes the projection device 11 to project the projection image 11b onto the projection area 11a on the projection surface 6a.

[0070] Next, the control device 14 acquires a captured image of the imaging range 12a from the imaging device 12 (step S12). For example, in step S12, the control device 14 instructs the imaging device 12 to capture an image and acquires the captured image from the imaging device 12. Alternatively, the imaging device 12 may repeatedly capture images of the imaging range 12a, and the control device 14 may acquire from the imaging device 12 the captured image obtained by capturing an image at the timing of step S12.

[0071] Next, the control device 14 performs a process of detecting the indicator marker 61 from the captured image acquired in step S12 (step S13). The detection process of step S13 is performed by, for example, image recognition processing based on the captured image.

[0072] Next, the control device 14 determines whether or not the indication marker 61 has been detected by the detection process of step S13 (step S14). If the indication marker 61 has not been detected (step S14: No), the control device 14 returns to step S12.

[0073] If the control device 14 detects the indication marker 61 in step S14 (step S14: Yes), the control device 14 changes a partial area of ​​the projection image 11b to be projected by the projection device 11 based on the position of the detected indication marker 61 (step S15), and the process returns to step S12. A specific example of changing a partial area of ​​the projection image 11b based on the position of the indication marker 61 will be described below.

[0074] <Changing a Partial Region of Projected Image 11b Based on the Position of Indication Marker 61> Fig. 8 is a diagram showing a first example of changing a partial region of the projected image 11b based on the position of the indication marker 61. In step S15 shown in Fig. 7, the control device 14 performs processing to reduce the visibility of the indication marker 61 on a partial region of the projected image 11b, for example.

[0075] Specifically, the control device 14 identifies a portion of the projection image 11b projected by the projection device 11 that is projected onto the projection surface 6a and overlaps with the indication marker 61 (hereinafter referred to as the "marker overlap area").

[0076] For example, the control device 14 identifies the marker superimposition area in the projection image 11b based on the area of ​​the indication marker 61 in the captured image detected by the detection process in step S13 and information indicating the positional relationship between the projection area 11a and the imaging range 12a. The information indicating the positional relationship between the projection area 11a and the imaging range 12a is acquired based on, for example, the positional relationship between the projection device 11 and the imaging device 12 in the moving body 10, the projection parameters of the projection device 11, and the projection parameters of the imaging device 12.

[0077] Then, the control device 14 performs image processing on the marker superimposition region in the projection image 11b to reduce the visibility of the indication marker 61 by overlapping with the indication marker 61 on the projection surface 6a. For example, the control device 14 can reduce the visibility of the indication marker 61 by changing the color of the marker superimposition region to a color that appears the same as or similar to the color of the surrounding area of ​​the marker superimposition region by overlapping with the indication marker 61.

[0078] As an example, assume that the area surrounding the marker superimposed area in the projection image 11b is achromatic (e.g., gray) and the color of the indicator marker 61 is red. In this case, by performing image processing to change the color of the marker superimposed area in the projection image 11b to blue-green, which is the complementary color of red, the marker superimposed area on which the indicator marker 61 is superimposed also appears achromatic (e.g., gray), and the visibility of the indicator marker 61 on the projection surface 6a can be reduced.

[0079] In this way, the control device 14 instructs the projection device 11 to project the projection image 11b (first image) onto the projection surface 6a, the imaging device 12 acquires a captured image (second image) obtained by capturing an image of the projection surface 6a, detects a specific indication marker 61 from the captured image, and modifies a partial area (e.g., a marker superimposition area) in the projection image 11b based on the position of the indication marker 61. This makes it possible to control the visibility of the indication marker 61 displayed on the projection surface 6a together with the projection image 11b. For example, the visibility of the indication marker 61 can be reduced to suppress the influence of the indication marker 61 on the visibility of the projection image 11b.

[0080] <Second Example of Changing a Partial Region of Projected Image 11b Based on the Position of Indication Marker 61> Fig. 9 is a diagram showing a second example of changing a partial region of the projected image 11b based on the position of the indication marker 61. In step S15 shown in Fig. 7, the control device 14 may perform processing to increase the visibility of the indication marker 61 on a partial region of the projected image 11b, for example.

[0081] Specifically, the control device 14 performs image processing on the marker superimposition area in the projection image 11b to overlap the indicator marker 61 on the projection surface 6a, thereby increasing the visibility of the indicator marker 61. For example, the control device 14 makes the brightness of the marker superimposition area lower than the brightness of the area surrounding the marker superimposition area (for example, sets the brightness to 0).

[0082] This reduces the effect on visibility caused by overlapping of the indication marker 61 with the projected image 11b, and increases the visibility of the indication marker 61, as shown in Fig. 9. Increasing the visibility of the indication marker 61 allows an observer, such as the user U, to easily recognize the current position of the indication marker 61.

[0083] <Third Example of Changing a Partial Area of ​​Projected Image 11b Based on the Position of Indication Marker 61> Fig. 10 is a diagram showing a third example of changing a partial area of ​​the projected image 11b based on the position of the indication marker 61. In step S15 shown in Fig. 7, in order to improve the visibility of the indication marker 61, the control device 14 may perform image processing to change the peripheral area of ​​the marker superimposition area of ​​the projected image 11b (a partial area based on the position of the indication marker 61).

[0084] For example, as shown in Fig. 10, the control device 14 may superimpose an enhanced image 11c indicating the marker superimposition area on the projection image 11b. In the example of Fig. 10, the enhanced image 11c is a circular image surrounding the marker superimposition area. However, the enhanced image 11c is not limited to a circular image, and may be an image of one or more arrows indicating the marker superimposition area. This emphasizes the position of the marker superimposition area, thereby improving the visibility of the indicator marker 61.

[0085] <Fourth Example of Changing a Partial Area of ​​Projected Image 11b Based on the Position of Indication Marker 61> Fig. 11 is a diagram showing a fourth example of changing a partial area of ​​the projected image 11b based on the position of the indication marker 61. In step S15 shown in Fig. 7, in order to improve the visibility of the indication marker 61, the control device 14 may perform image processing to change both the marker superimposed area of ​​the projected image 11b and the area surrounding the marker superimposed area of ​​the projected image 11b.

[0086] For example, as shown in Fig. 11, the control device 14 makes the brightness of the marker superimposed area of ​​the projection image 11b and the brightness of the area surrounding the marker superimposed area of ​​the projection image 11b lower than those of other areas of the projection image 11b (for example, sets the brightness to 0). This reduces the effect on visibility caused by the overlap of the indication marker 61 with the projection image 11b as in the example of Fig. 9, and also makes it possible to point to the marker superimposed area as in the example of Fig. 10, thereby improving the visibility of the indication marker 61.

[0087] <Fifth Example of Changing a Partial Area of ​​Projected Image 11b Based on the Position of Indication Marker 61> Fig. 12 is a diagram showing a fifth example of changing a partial area of ​​the projected image 11b based on the position of the indication marker 61. In step S15 shown in Fig. 7, the control device 14 may superimpose and display a specific symbol image on the marker superimposition area in the projected image 11b.

[0088] For example, the control device 14 may superimpose a symbol image 11d on the projection image 11b at the position of the marker superimposition area as shown in Fig. 12. In the example of Fig. 12, the symbol image 11d is an image of a person's finger. For example, the control device 14 superimposes and displays the symbol image 11d so that the reference position (e.g., the center position) of the marker superimposition area coincides with the reference position of the symbol image 11d (e.g., the position of the fingertip).

[0089] This allows an observer such as the user U to easily recognize the current position of the instruction marker 61, just as in the case where the visibility of the instruction marker 61 itself is increased. Furthermore, since the symbol image 11d is an image projected by the projection device 11, it can be a complex image that is difficult to display as an instruction marker 61 using the laser pointer 60. This makes it possible to more easily recognize the current position of the instruction marker 61. Furthermore, it is also possible to obtain dramatic effects by displaying the symbol image 11d.

[0090] 12, the control device 14 performs processing to reduce the visibility of the indicator marker 61, similar to the example of Fig. 8. This reduces the effect on visibility caused by overlapping of the symbol image 11d with the indicator marker 61, and can increase the visibility of the symbol image 11d.

[0091] <Movement of Symbol Image 11d in Accordance with Movement of Indicator Marker 61> Fig. 13 is a diagram showing an example of movement of the symbol image 11d in response to movement of the indicator marker 61. By the process (repeated process) of Fig. 7 , the position of the symbol image 11d moves in accordance with the change in the position of the indicator marker 61. For example, as shown in Fig. 13 , when the user U operates the laser pointer 60 to move the indicator marker 61 to the right, the indicator marker 61 also moves in the right direction in accordance with the movement of the indicator marker 61.

[0092] In this way, the control device 14 changes the position of the symbol image 11d in the projection image 11b in accordance with the change in the position of the indication marker 61. As a result, even if the user U moves the indication marker 61, an observer such as the user U can easily recognize the position of the indication marker 61 after the movement by the symbol image 11d.

[0093] <Non-Following of the Symbol Image 11d Due to Shaking of the Pointer Marker 61, etc.> Fig. 14 is a diagram showing an example of non-following of the symbol image 11d due to shaking of the pointer marker 61, etc. For example, the control device 14 may perform low-pass filtering on the detection result of the position of the pointer marker 61, and cause the symbol image 11d to follow the detection result of the low-pass filtering. In this way, even if the pointer marker 61 is slightly shaken due to shaking of the laser pointer 60 held by the user U, for example, as in the example of Fig. 14 , it is possible to make it difficult for the symbol image 11d to follow the shaking of the pointer marker 61.

[0094] In this way, when the change in the position of the indicator marker 61 satisfies a predetermined condition (for example, when the indicator marker 61 vibrates finely at a high frequency), the control device 14 may not change the position of the symbol image 11d in the projection image 11b in accordance with the change in the position of the indicator marker 61. This makes it possible to suppress blurring of the symbol image 11d caused by camera shake or the like, and improve the projection quality.

[0095] <Changing Predetermined Conditions Based on the Size, etc., of the Detected Indicator Marker 61> FIG. 15 is a diagram showing an example of changing the predetermined conditions based on the size, etc., of the detected indicator marker 61. In the example of FIG. 15, the indicator marker 61 in the imaging range 12a is larger than in the example of FIG. 14. In such a case, it can be estimated that the laser pointer 60 is located farther from the projection surface 6a than in the example of FIG. 14. In this case, the control device 14 may extend the predetermined condition for changing the position of the indicator marker 61 so as not to change the position of the symbol image 11d in the projection image 11b, compared to the example of FIG. 14.

[0096] For example, the larger the detected pointing marker 61 is, that is, the further away the laser pointer 60 is estimated to be from the projection surface 6a, the stronger the low-pass filtering process the control device 14 performs on the detection results of the position of the pointing marker 61.

[0097] As a result, the further the laser pointer 60 is positioned on the projection surface 6a and the greater the shaking of the indicator marker 61 due to camera shake or the like, the lower the sensitivity of the movement of the symbol image 11d to the movement of the indicator marker 61, thereby making it possible to strongly suppress the shaking of the symbol image 11d due to camera shake or the like.

[0098] Furthermore, the closer the laser pointer 60 is positioned to the projection surface 6a and the smaller the shaking of the indicator marker 61 caused by hand shake or the like, the higher the sensitivity of the movement of the symbol image 11d to the movement of the indicator marker 61, thereby improving the ability of the symbol image 11d to follow the intentional movement of the indicator marker 61.

[0099] Furthermore, the higher the brightness of the pointing marker 61 in the imaging range 12a, the closer the laser pointer 60 is estimated to be located on the projection surface 6a. Therefore, the control device 14 may perform a stronger low-pass filtering process on the detection result of the position of the pointing marker 61 as the brightness of the detected pointing marker 61 decreases, i.e., as the laser pointer 60 is estimated to be located farther from the projection surface 6a.

[0100] In addition, the control device 14 may combine the size and brightness of the indication marker 61 and perform low-pass filtering processing on the detection result of the position of the indication marker 61, the stronger the larger the detected indication marker 61, and the stronger the lower the brightness of the detected indication marker 61.

[0101] In this way, the control device 14 may change the specified conditions for changing the position of the indication marker 61 so as not to change the position of the symbol image 11d in the projected image 11b, based on at least one of the size and brightness of the detected indication marker 61.

[0102] <Changing the Size of the Symbol Image 11d Based on the Size, etc., of the Detected Indicator Marker 61> Figure 16 is a diagram showing an example of changing the size of the symbol image 11d based on the size, etc., of the detected indicator marker 61. In the example of Figure 16, the indicator marker 61 in the imaging range 12a is larger than in the example of Figure 13, etc. In such a case, it can be estimated that the laser pointer 60 and the user U are located farther from the projection surface 6a than in the example of Figure 13, etc. In this case, the control device 14 may increase the size of the symbol image 11d in the projection image 11b compared to the example of Figure 13, etc.

[0103] For example, the control device 14 increases the size of the symbol image 11d as the detected indication marker 61 increases, i.e., as the user U is estimated to be located farther from the projection surface 6a. As a result, the size of the symbol image 11d increases as the user U is located farther from the projection surface 6a and the symbol image 11d becomes more difficult to view, thereby preventing deterioration in visibility of the symbol image 11d.

[0104] Furthermore, the higher the brightness of the instruction marker 61 in the imaging range 12a, the closer it can be estimated that the laser pointer 60 and the user U are located on the projection surface 6a. Therefore, the control device 14 may increase the size of the symbol image 11d as the brightness of the detected instruction marker 61 decreases, i.e., as it is estimated that the user U is located farther away from the projection surface 6a.

[0105] Furthermore, the control device 14 may combine the size and brightness of the indicator marker 61 and increase the size of the symbol image 11d as the detected indicator marker 61 increases and as the brightness of the detected indicator marker 61 decreases. In this way, the control device 14 may change the size of the symbol image 11d in the projection image 11b based on at least one of the size and brightness of the detected indicator marker 61.

[0106] <Display of the trajectory of movement of the symbol image 11d> Fig. 17 is a diagram showing an example of the display of the trajectory of movement of the symbol image 11d. The control device 14 may superimpose a trajectory image 11e on the projection image 11b. The trajectory image 11e is an image that represents the trajectory of the change in position of the symbol image 11d in the projection image 11b.

[0107] For example, the control device 14 generates and stores history data on the position of the symbol image 11d in the projection image 11b (for example, the position of the fingertip or the center position), and generates the trajectory image 11e based on this history data.The control device 14 then superimposes the trajectory image 11e on the projection image 11b together with the symbol image 11d.This allows an observer such as the user U to easily recognize changes in the position of the symbol image 11d.

[0108] In addition, the control device 14 generates and stores historical data of changes in the symbol image 11d in the projection image 11b, making it possible to later replay and display the changes in the symbol image 11d in the projection image 11b based on the historical data, or to use the historical data for data analysis, machine learning, etc.

[0109] 18 is a diagram showing an example of limiting the direction of change in the position of the symbol image 11d in the projection image 11b. The control device 14 may perform control to limit the direction of change in the position of the symbol image 11d in the projection image 11b.

[0110] A marker trajectory 181 shown in Fig. 18 is a virtual illustration of the trajectory of movement of the indicator marker 61. In the example of the marker trajectory 181 in Fig. 18, the indicator marker 61 moves to the right while meandering.

[0111] In contrast to this, for example, the control device 14 limits the direction of movement of the symbol image 11d to only the horizontal (lateral) direction. Specifically, the control device 14 causes the horizontal position of the symbol image 11d to follow changes in the horizontal position of the indicator marker 61, but does not cause the position of the symbol image 11d to follow changes in the vertical position of the indicator marker 61. The symbol image trajectory 182 is a virtual illustration of the trajectory of movement of the symbol image 11d.

[0112] As a result, in a situation where the symbol image 11d only needs to be moved horizontally, even if the indication marker 61 shakes vertically due to camera shake or the like, the shaking is not reflected in the symbol image 11d, making it easier to move the symbol image 11d.

[0113] <Change of symbol image 11d based on change in position of indicator marker 61> Fig. 19 is a diagram showing an example of change of the symbol image 11d based on change in position of the indicator marker 61. The control device 14 may change the symbol image 11d based on change in position of the indicator marker 61 in the projection image 11b.

[0114] 19, when the instruction marker 61 moves leftward, the control device 14 moves the symbol image 11d leftward following the instruction marker 61 and replaces the symbol image 11d with an image of a finger pointing left. When the instruction marker 61 moves rightward, the control device 14 moves the symbol image 11d rightward following the instruction marker 61 and replaces the symbol image 11d with an image of a finger pointing right.

[0115] This allows an observer such as the user U to more intuitively recognize the position and movement direction of the indication marker 61.

[0116] <Maintaining the Position of the Symbol Image 11d When the Pointing Marker 61 Cannot Be Detected> Fig. 20 is a diagram showing an example of maintaining the position of the symbol image 11d when the pointing marker 61 cannot be detected. In the example of Fig. 20, the left end of the projection area 11a is outside the imaging range 12a. Then, assume that the pointing marker 61 moves from near the center of the projection area 11a to the upper left, moving to a position that is included in the projection area 11a but not included in the imaging range 12a.

[0117] In this case, the control device 14 moves the symbol image 11d in accordance with the movement of the indicator marker 61, but when the indicator marker 61 leaves the imaging range 12a, it becomes unable to detect the indicator marker 61. When the control device 14 becomes unable to detect the indicator marker 61, it holds the position of the symbol image 11d at that time. Thereafter, when the indicator marker 61 returns to the imaging range 12a and becomes detectable, the control device 14 resumes the process of moving the symbol image 11d in accordance with the movement of the indicator marker 61.

[0118] This allows an observer such as the user U to easily recognize that the indication marker 61 has left the image capture range 12a and the position at which the indication marker 61 has left the image capture range 12a.

[0119] <Inability to detect indicator marker 61> Fig. 21 is a diagram showing an example of a displayed image indicating that an indicator marker 61 cannot be detected. In the example shown in Fig. 20, when an indicator marker 61 that had been detected up until then becomes undetectable, the control device 14 may superimpose a notification image 211 indicating that the indicator marker 61 cannot be detected on the projection image 11b, as shown in Fig. 21 .

[0120] <Display of symbol image 11d when position of symbol image 11d cannot be changed> Fig. 22 is a diagram showing an example of display of symbol image 11d when position of symbol image 11d cannot be changed. For example, as shown in Fig. 22, it is assumed that the indication marker 61 moves from near the center of the projection area 11a to the upper left and then to the outside of the projection area 11a.

[0121] In this case, the control device 14 moves the symbol image 11d in accordance with the movement of the indicator marker 61, but midway through, it becomes impossible to move the symbol image 11d in accordance with the movement of the indicator marker 61. In this case, the control device 14 displays the symbol image 11d at a position corresponding to the position of the indicator marker 61 within the area in the projection image 11b where the symbol image 11d can be displayed.

[0122] For example, the control device 14 displays the symbol image 11d at a position closest to the indication marker 61 on a line connecting the indication marker 61 and the center of the projection image 11b within the area in the projection image 11b where the symbol image 11d can be displayed.

[0123] This makes it possible for an observer such as user U to easily recognize that the indication marker 61 has moved outside the projection area 11a and the symbol image 11d can no longer follow it, and the approximate position of the indication marker 61 that has moved outside the projection area 11a.

[0124] <Changing the symbol image 11d based on the state of the indicator marker 61> For example, it is assumed that the state of the indicator marker 61 can be changed by operating the laser pointer 60. The state of the indicator marker 61 includes, for example, at least one of the color of the indicator marker 61, the shape of the indicator marker 61, the size of the indicator marker 61, and the blinking cycle of the indicator marker 61.

[0125] In this case, the control device 14 may change the symbol image 11d based on the state of the indication marker 61. This allows the user U to change the state of the indication marker 61 by operating the laser pointer 60 or the like in accordance with the environment around the projection surface 6a, the content of the projection image 11b, and the like, thereby changing the symbol image 11d.

[0126] <Display of symbol image 11d at a position different from the position of the indication marker 61> Fig. 23 is a diagram showing an example of display of the symbol image 11d at a position different from the position of the indication marker 61. In the example of Fig. 23, the center of the projection area 11a is shifted upward with respect to the center of the imaging range 12a, and the upper part of the projection area 11a is outside the imaging range 12a.

[0127] In such a case, the control device 14 may display the symbol image 11d so that the relative position of the symbol image 11d in the projection image 11b is similar to the relative position of the indicator marker 61 in the imaging range 12a, rather than displaying the symbol image 11d at the position of the indication marker 61 in the projection image 11b. This allows the symbol image 11d to be displayed by an intuitive operation even in an area of ​​the projection area 11a that is outside the imaging range 12a, as in the example of Fig. 23 .

[0128] 23, the control device 14 performs processing to reduce the visibility of the indication marker 61, similar to the example of Fig. 8. As a result, even when the symbol image 11d is displayed at a position different from the position of the indication marker 61, the visibility of the indication marker 61 can be reduced, and the influence of the indication marker 61 on the visibility of the projected image 11b can be suppressed.

[0129] In this way, the control device 14 may display the symbol image 11d at a position different from the position of the indication marker 61 in the projection image 11b, depending on the difference between the projection area 11a (first area) on the projection surface 6a where the projection image 11b is projected and the imaging range 12a (second area) on the projection surface 6a where the indication marker 61 can be detected. In this case, when the indication marker 61 is present in the projection area 11a, the control device 14 may change a part of the area in the projection image 11b based on the position of the indication marker 61 so as to reduce the visibility of the indication marker 61.

[0130] <Another Example of Processing by Control Device 14> Figure 24 is a flowchart showing another example of processing by the control device 14. The control device 14 executes, for example, the processing shown in Figure 24. Steps S21 to S24 shown in Figure 24 are similar to steps S11 to S14 shown in Figure 7.

[0131] In step S24, if the control device 14 detects the indication marker 61 (step S24: Yes), the control device 14 changes the projection area 11a of the projection device 11 based on the detected indication marker 61 (step S25). The change of the projection area 11a of the projection device 11 based on the indication marker 61 will be described later (see, for example, FIG. 25).

[0132] Next, the control device 14 changes a part of the area based on the detected position of the indicator marker 61 (step S26), and returns to step S22. The process of step S26 is the same as the process of step S15 shown in FIG.

[0133] <Changing the Position of the Projection Region 11a by Moving the Indication Marker 61> Figure 25 is a diagram showing an example of changing the position of the projection region 11a by moving the indication marker 61. In step S25 shown in Figure 24, the control device 14 changes the position of the projection region 11a, for example, based on the position of the indication marker 61 on the projection surface 6a. For example, the control device 14 changes the position of the projection region 11a so that the center position of the projection region 11a coincides with the position of the indication marker 61.

[0134] The position of the projection area 11a may be changed by instructing the projection device 11 to perform the various shifts and projection direction changes described above, or by instructing the moving mechanism 16 to change at least one of the position and attitude of the moving body 10, or by a combination of these.

[0135] For example, in the state shown in Fig. 6, suppose that the user U operates the laser pointer 60 to move the position of the indication marker 61 downward as shown in Fig. 25. In this case, the control device 14 moves the projection area 11a downward so that the center position of the projection area 11a coincides with the position of the moved indication marker 61. In the example of Fig. 25, the control device 14 moves the moving object 10 downward to move the projection area 11a downward.

[0136] For example, the control device 14 acquires information indicating the positional relationship between the projection area 11a and the imaging range 12a, and information indicating the positional relationship (e.g., distance) between the projection device 11 and the imaging device 12 and the projection surface 6a. The information on the positional relationship between the projection device 11 and the imaging device 12 and the projection surface 6a is acquired, for example, by a distance measuring means provided in the mobile object 10. Based on this information, the control device 14 calculates control parameters (e.g., the direction and amount of movement of the mobile object 10) for matching the center position of the projection area 11a with the position of the indication marker 61, and performs control to move the projection area 11a using the calculated control parameters.

[0137] This allows the user U to operate the laser pointer 60 to move the indication marker 61, thereby moving the projection area 11a.

[0138] Furthermore, the control device 14 may change the position of the projection region 11a based not only on the position of the detected indication marker 61 but also on the state of the detected indication marker 61. As described above, the state of the indication marker 61 includes, for example, at least one of the color of the indication marker 61, the shape of the indication marker 61, the size of the indication marker 61, and the blinking cycle of the indication marker 61. As an example, the control device 14 changes the position of the projection region 11a based on the color of the detected indication marker 61.

[0139] Furthermore, the control device 14 may change not only the position of the projection area 11a but also the size of the projection area 11a based on the detected position and state of the indication marker 61. As an example, the control device 14 increases the size of the projection area 11a when it detects upward movement of the indication marker 61, and decreases the size of the projection area 11a when it detects downward movement of the indication marker 61. The size of the projection area 11a may be changed by instructing the projection device 11 to optically or electronically enlarge or reduce the size, by instructing the movement mechanism 16 to change the position of the moving body 10, or by a combination of these.

[0140] (Modifications) <Modifications of the indicator marker 61> Although the case where the indicator marker 61 is formed by visible light projected onto the projection surface 6a has been described, the indicator marker 61 is not limited to this, and may be formed by invisible light (for example, infrared light) projected onto the projection surface 6a as long as it can be detected based on the captured image obtained by the imaging device 12. In this case, the influence of the indicator marker 61 on the visibility of the projection image 11b can be suppressed without performing processing for reducing the visibility of the indicator marker 61 as described with reference to FIG. 8, for example.

[0141] Furthermore, although the case where the instruction marker 61 is the laser pointer 60 held by the user U has been described, the instruction marker 61 is not limited to this. For example, the instruction marker 61 may be light emitted from an irradiation device located in a different location from the user U. Furthermore, the instruction marker 61 may be an object (for example, a mobile robot) that can move on the projection surface 6 a.

[0142] <Variations in the method of changing the projected image 11b> We have explained the case where a portion of the projected image 11b is changed based on the position of the indication marker 61, but if the user U is able to control the color, shape, size, blinking cycle, etc. of the indication marker 61, the control device 14 may also change a portion of the projected image 11b based on the color, shape, size, blinking cycle, etc. of the indication marker 61.

[0143] <Blinking of Projected Image 11b> The control device 14 may instruct the projection device 11 to blink the projected image 11b. This allows the control device 14 to accurately detect the indication marker 61 based on the image captured by the imaging device 12 at a timing when the projected image 11b is not displayed. It is desirable that the blinking of the projected image 11b be performed within a range that does not significantly affect the observation of the projected image 11b by an observer such as the user U.

[0144] Furthermore, the laser pointer 60 may emit a blinking light, causing the indicator marker 61 to blink on the projection surface 6a. In this case, the control device 14 may identify the timing of the blinking of the indicator marker 61 based on the image captured by the imaging device 12, and may hide the projection image 11b during the period when the indicator marker 61 is displayed, and display the projection image 11b during the period when the indicator marker 61 is not displayed. This allows the indicator marker 61 to be displayed during the period when the projection image 11b is not displayed, making it possible to accurately detect the indicator marker 61, and prevents the indicator marker 61 from being displayed during the period when the projection image 11b is displayed, thereby preventing deterioration in visibility of the projection image 11b due to the indicator marker 61.

[0145] <Other Examples of Mobile Body 10> Although the mobile body 10 has been described as a multicopter, the mobile body 10 may be an aircraft (flying object) other than a multicopter. Furthermore, the mobile body 10 is not limited to a flying object, and may be a vehicle or robot that runs or walks on the ground. Furthermore, the projection device 11, the imaging device 12, and the control device 14 do not have to be mounted on the mobile body 10.

[0146] <Other Examples of Projection Device 11> Although the configuration in which the projection device 11 is mounted on the moving body 10 has been described, the present invention is not limited to this configuration. For example, the projection device 11 may be a projection device fixed on the ground, or may be a projection device provided in the information terminal 40.

[0147] <Other Examples of Imaging Device 12> Although the configuration in which the imaging device 12 is mounted on the moving body 10 has been described, the present invention is not limited to this configuration. For example, the imaging device 12 may be an imaging device fixed on the ground, or an imaging device provided in the information terminal 40.

[0148] <Other Examples of Control Device> Although the case where the control device of the present invention is applied to the control device 14 of the mobile body 10 has been described, the present invention is not limited to this configuration. The control device of the present invention may be applied to, for example, an information terminal 40. In this case, the information terminal 40 communicates with the mobile body 10 to perform various controls similar to those performed by the control device 14 described above.

[0149] Although various embodiments have been described above, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention. Furthermore, the components of the above embodiments may be combined in any manner as long as they do not deviate from the spirit of the invention.

[0150] This application is based on a Japanese patent application (Patent Application No. 2022-081603) filed on May 18, 2022, the contents of which are incorporated by reference into this application.

[0151] 6 Projection object 6a Projection surface 10 Moving body 11 Projection device 11a Projection area 11b Projected image 11c Enhanced image 11d Symbol image 11e Trajectory image 12 Imaging device 12a Imaging range 14 Control device 14a Storage medium 15 Communication unit 16 Moving mechanism 31 Light source 32 Light modulation unit 33 Projection optical system 34 Control circuit 40 Information terminal 41 Processor 42 Memory 43 Communication interface 44 User interface 49 Bus 60 Laser pointer 61 Pointing marker 181 Marker trajectory 182 Symbol image trajectory 211 Notification image

Claims

1. A control device including a processor, The processor, instructing a projection device to project a first image onto a projection surface; an imaging device captures an image of the projection surface to obtain a second image; Detecting a specific marker from the second image; A part of an area based on the position of the marker in the first image is changed so as to reduce the visibility of the marker. Control device.

2. The control device according to claim 1, The processor displays a specific symbol image in the partial area of ​​the first image. Control device.

3. The control device according to claim 2, The processor changes a position of the symbol image in the first image in response to a change in the position of the marker. Control device.

4. A control device as claimed in claim 3, When the change in the position of the marker satisfies a predetermined condition, the processor does not change the position of the symbol image in the first image in response to the change in the position of the marker. Control device.

5. A control device as claimed in claim 4, The processor changes the predetermined condition based on at least one of a size and a brightness of the detected marker. Control device.

6. The control device according to claim 4, The processor changes a size of the symbol image in the first image based on at least one of a size and a brightness of the detected marker. Control device.

7. A control device as claimed in claim 3, The processor displays, on the first image, an image representing a trajectory of a change in the position of the symbol image in the first image. Control device.

8. A control device as claimed in claim 3, the processor generates historical data of changes in the symbol image in the first image; Control device.

9. A control device as claimed in claim 3, The processor limits a direction of change in the position of the symbol image in the first image. Control device.

10. The control device according to claim 3, The processor modifies the symbol image based on a change in a position of the marker in the first image. Control device.

11. A control device as claimed in claim 3, The processor holds the position of the symbol image when the marker that was detected becomes undetectable. Control device.

12. A control device as claimed in claim 3, When the position of the symbol image in the first image cannot be changed in response to the change in the position of the marker, the processor displays the symbol image at a position corresponding to the position of the marker within an area in the first image where the symbol image can be displayed. Control device.

13. A control device as claimed in claim 2, The processor modifies the symbol image based on a state of the marker. Control device.

14. The control device according to claim 13, The state of the marker includes at least one of a color of the marker, a shape of the marker, a size of the marker, and a cycle of blinking of the marker. Control device.

15. The control device according to claim 2, The processor, displaying the symbol image at a position different from a position of the marker in the first image according to a difference between a first area on the projection surface onto which the first image is projected and a second area on the projection surface in which the marker can be detected; When the marker is present in the first region, the partial region is changed so as to reduce the visibility of the marker. Control device.

16. The control device according to claim 1, When the marker that has been detected becomes undetectable, the processor displays an image indicating that the marker is undetectable on the first image. Control device.

17. The control device according to claim 1, The marker is an invisible light projected onto the projection surface. Control device.

18. The control device according to claim 1, The processor instructs the projection device to change at least one of a position and a size of a projection area in response to a detection result of the marker. Control device.

19. The control device according to claim 1, The processor instructs the projection device to blink the first image. Control device.

20. A moving body comprising the control device according to any one of claims 1 to 19, the projection device, and the imaging device, The control device is capable of controlling the movement of the moving body. Mobile body.

21. A moving body according to claim 20, the processor performs the movement control to change at least one of a position and a size of a projection area of ​​the projection device in response to a detection result of the marker. Mobile body.

22. A processor comprising: instructing a projection device to project a first image onto a projection surface; an imaging device captures an image of the projection surface to obtain a second image; Detecting a specific marker from the second image; A part of an area based on the position of the marker in the first image is changed so as to reduce the visibility of the marker. Control methods.

23. A processor comprising: instructing a projection device to project a first image onto a projection surface; an imaging device captures an image of the projection surface to obtain a second image; Detecting a specific marker from the second image; A part of an area based on the position of the marker in the first image is changed so as to reduce the visibility of the marker. A control program for executing processing.

24. A control device comprising a processor, The processor, instructing a projection device to project a first image onto a projection surface; an imaging device captures an image of the projection surface to obtain a second image; Detecting a specific marker from the second image; A part of the first image is changed based on the position of the marker; Displaying a specific symbol image in the partial area of ​​the first image; The processor changes the symbol image based on a state of the marker including at least one of a shape of the marker, a size of the marker, and a cycle of blinking of the marker. Control device.

25. A processor comprising: instructing a projection device to project a first image onto a projection surface; an imaging device captures an image of the projection surface to obtain a second image; Detecting a specific marker from the second image; A part of the first image is changed based on the position of the marker; Displaying a specific symbol image in the partial area of ​​the first image; The processor changes the symbol image based on a state of the marker including at least one of a shape of the marker, a size of the marker, and a cycle of blinking of the marker. Control methods.

26. A processor comprising: instructing a projection device to project a first image onto a projection surface; an imaging device captures an image of the projection surface to obtain a second image; Detecting a specific marker from the second image; A part of the first image is changed based on the position of the marker; Displaying a specific symbol image in the partial area of ​​the first image; The processor changes the symbol image based on a state of the marker including at least one of a shape of the marker, a size of the marker, and a cycle of blinking of the marker. A control program for executing processing.