Imaging device, imaging system, imaging method, and computer program

Abstract

To provide an imaging device that can suppress image quality degradation while understanding the surrounding environment. [Solution] The imaging device is characterized by comprising: an imaging means for capturing an optical image formed by an optical system; and an illumination means for irradiating an illumination light in front of the imaging means and for notifying imaging information related to the imaging means by controlling the projection shape and position of the illumination light.

Description

Technical Field

[0001] The present invention relates to an imaging device, an imaging system, an imaging method, a computer program, and the like.

Background Art

[0002] In recent years, with the popularity of video posting sites, action cameras that are easy to use, tough, and can capture impressive videos for outdoor activities, sports, etc. depending on how they are used have attracted attention. On the other hand, single-lens reflex cameras are becoming more and more high-definition and high-resolution. Against this background, the demand for usage such as walking shooting and running shooting has been increasing even in single-lens reflex cameras.

[0003] Patent Document 1 discloses a technique in which a projection optical device is provided in front of a camera, and light of a certain color is irradiated within the shooting angle range, so that the photographer and the subject can easily recognize the shooting angle range.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0005] However, in a single-lens reflex camera that emphasizes video quality, shooting while holding it with both hands and checking framing etc. with a live view image is assumed. Therefore, there is concern about the risk of the photographer falling, etc., but on the other hand, if too much attention is paid to the surrounding situation, framing will be neglected, and the quality of the video will deteriorate.

[0006] On the other hand, in the conventional technology disclosed in the above Patent Document 1, since the irradiation light hits within the shooting angle, there is a problem that, for example, the image quality deteriorates.

[0007] This invention has been made in view of the above-mentioned problems, and aims to provide an imaging device that can suppress the deterioration of image quality while understanding the surrounding environment. [Means for solving the problem]

[0008] To achieve the above objective, the imaging apparatus according to an embodiment of the present invention is An imaging means for capturing an optical image formed by an optical system, An illumination means that emits illumination light in front of the imaging means and controls the projection shape and position of the illumination light to notify imaging information related to imaging by the imaging means, An imaging device characterized by having the following features. [Effects of the Invention]

[0009] According to the present invention, it is possible to create an imaging device that can suppress the deterioration of image quality while understanding the surrounding environment. [Brief explanation of the drawing]

[0010] [Figure 1] This is a functional block diagram showing an example configuration of a digital camera 100 as an imaging device according to Embodiment 1 of the present invention. [Figure 2] This flowchart shows the process of displaying captured information by visible light irradiation in Embodiment 1. [Figure 3] This is a schematic diagram illustrating the principle of controlling the irradiation angle of the irradiated light. [Figure 4] (A) is a schematic diagram illustrating the method for calculating the shooting angle of view, and (B) is a schematic diagram illustrating the relationship between the orientation of the digital camera 100 and the lower edge of the shooting angle of view. [Figure 5] This is a schematic diagram illustrating the method for displaying captured information by visible light irradiation according to Embodiment 1. [Figure 6] (A) and (B) are illustrative diagrams showing examples of image information display by visible light irradiation in Embodiment 1. [Figure 7] This is a functional block diagram showing an example configuration of AR glasses 700, which are a head-mounted display device according to Embodiment 2. [Figure 8] It is a schematic external view of the AR glasses 700 which is a head-mounted display device of Embodiment 2. [Figure 9] It is a flowchart showing an example of photographing information display processing by non-visible light irradiation in the imaging system according to Embodiment 2. [Figure 10] It is a schematic diagram showing a method for displaying photographing information by non-visible light irradiation in Embodiment 2. [Figure 11] (A) and (B) are image diagrams showing examples of photographing information display by non-visible light irradiation in Embodiment 2. [Figure 12] It is a flowchart showing an example of photographing information display processing by visible light irradiation in Embodiment 3. [Figure 13] It is a diagram showing an example of control timing for photographing information display by visible light irradiation in Embodiment 3. [Figure 14] It is a flowchart showing an example of display processing for photographing information and photographing-related information by visible light irradiation in Embodiment 4. [Figure 15] It is a schematic diagram showing a method for displaying photographing information and photographing-related information by visible light irradiation in Embodiment 4. [Figure 16] It is a flowchart showing an example of display processing for photographing information and photographing-related information by non-visible light irradiation in Embodiment 5. [Figure 17] It is a schematic diagram showing a method for displaying photographing information and photographing-related information by non-visible light irradiation in Embodiment 5. [Figure 18] It is an image diagram showing an example of photographing-related information display by non-visible light irradiation in Embodiment 5.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each figure, the same members or elements are given the same reference numerals, and duplicate descriptions are omitted or simplified.

[0012] <Embodiment 1> Figure 1 is a functional block diagram showing an example configuration of a digital camera 100 as an imaging device according to Embodiment 1 of the present invention. Note that some of the functional blocks shown in Figure 1 are realized by having the CPU, etc., which is a computer included in the imaging device, execute a computer program stored in the memory, which is a storage medium.

[0013] However, some or all of these can be implemented in hardware. Hardware options include dedicated circuits (ASICs) and processors (reconfigurable processors, DSPs).

[0014] Furthermore, each functional block shown in Figure 1 does not necessarily have to be housed in the same enclosure; they may be composed of separate devices connected to each other via signal paths. The above explanation regarding Figure 1 also applies to Figure 7.

[0015] In Figure 1, 101 is the imaging optical unit, which includes multiple lens groups, including a focusing lens and an image stabilization lens, as well as an aperture. During shooting, the imaging optical unit 101 adjusts the focus with the focus control unit 118, adjusts the exposure with the aperture control unit 119, corrects for blur, etc., and forms an optical image on the image sensor 102.

[0016] The image sensor 102 has a photoelectric conversion function that converts the optical image formed via the imaging optical unit 101 into an electrical signal (analog image signal), and is composed of a CCD or CMOS sensor, etc. Furthermore, the image sensor 102 is equipped with multiple independent photodiodes within each pixel for image plane phase-detection autofocus. The image sensor 102 functions as an imaging means that captures the optical image formed by the optical system.

[0017] The A / D conversion unit 103 converts the analog image signal from the image sensor 102 into a digital image signal. The converted image data is input to the subsequent image processing unit 104. Bus 116 is a system bus mainly used to transmit control signals from the CPU 114 and other components to each block, while bus 117 is a data bus mainly used to transfer image data.

[0018] The CPU 114, acting as a computer, controls the entire image processing unit, issues operational instructions to each functional block, and functions as a control means to execute various control processes.

[0019] Furthermore, the CPU 114 also performs calculations necessary for various control processes. The CPU 114 controls the image processing unit 104, data transfer unit 105, memory control unit 106, non-volatile memory control unit 108, recording media control unit 110, display control unit 112, operation unit 115, image sensor 102, etc., via the bus 116.

[0020] The CPU 114 implements each of the processes in this embodiment by executing the computer program recorded in the ROM 109. Furthermore, the CPU 114 also controls the lens and aperture of the imaging optical unit 101 and acquires information such as the focal length.

[0021] The data transfer unit 105 is composed of multiple DMACs (Direct Memory Access Controllers) that perform data transfer.

[0022] The DRAM (memory) 107 is a memory that stores data, such as a predetermined number of still images, a predetermined amount of video footage, audio, constants for the operation of the CPU 114, and computer programs. The memory control unit 106 writes data to and reads data from the DRAM 107 in response to instructions from the CPU 114 or the data transfer unit 105.

[0023] The non-volatile memory control unit 108 writes and reads data from the ROM (non-volatile memory) 109 in response to instructions from the CPU 114. The ROM 109 is an electrically erasable and recordable memory, and EEPROM or similar is used. The ROM 109 stores constants for the operation of the CPU 114, computer programs, and the like.

[0024] The image processing unit 104 consists of various image processing circuits and buffer memory, and performs processing such as chromatic aberration correction, development processing, noise reduction processing, and resizing on the image data converted by the A / D conversion unit 103. In addition, the image processing unit 104 performs pixel correction, black level correction, shading correction, and scratch correction, and also performs processing to detect, recognize, and track a subject from the developed and processed image.

[0025] The recording medium 111 is a recording medium such as an SD card, and is controlled by the recording medium control unit 110 to record image data and read recorded data.

[0026] The display unit 113 consists of a liquid crystal display and an electronic viewfinder, and is controlled by the display control unit 112. It displays various image data output from the image processing unit 104, menu screens, and the like. In addition, before taking still images and during video recording, it displays image data input from the A / D conversion unit 103 in real time.

[0027] The operating unit 115, which is an operating means, includes switches, buttons, touch panels, etc. that can be operated by the user, and is used for operations such as turning the power on / off and turning the shutter on / off.

[0028] The autofocus detection unit 120 detects the focus state of a predetermined area within the image using an image plane phase difference method based on the focus detection image output from the image sensor 102. It then calculates the amount of lens drive required to bring the predetermined area into focus and instructs the focus control unit 118 to control the drive of the focusing lens.

[0029] The electronic image stabilization unit 121 is located within the image processing unit 104 and performs image processing to suppress blur of the subject in the image acquired by the image sensor 102. The electronic image stabilization unit 121 functions as an electronic image stabilization means that performs electronic blur correction on the image captured by the imaging means.

[0030] The illumination light source unit 122 has a drive unit for controlling the emission and irradiation direction of a light source such as a laser, and is intended to project information related to shooting, such as the shooting angle of view, in front of the digital camera 100 to present information to the user. In this embodiment, the illumination light source unit 122 emits visible light.

[0031] The illumination control unit 123 generates control signals for turning the light source on and off, the illumination angle, and the angle change cycle, thereby controlling the illumination light source unit 122. The illumination light source unit 122 and the illumination control unit 123 function as illumination means that emit illumination light in front of the imaging means and control the projection shape and position of the illumination light to notify imaging information related to imaging by the imaging means. Details of the mechanism of the illumination light source unit 122 and the method of calculating the illumination angle of the illumination control unit 123 will be described later.

[0032] 124 is a communication unit and functions as a means of communication between the digital camera 100, which is an imaging device, and external terminals or AR glasses described later. The communication by the communication unit 124 may be wireless or wired.

[0033] Figure 2 is a flowchart showing the display process of captured information by visible light irradiation in Embodiment 1, and illustrates the display control process flow of captured information (e.g., line-shaped markers) by visible light irradiation of the digital camera 100, which serves as the imaging device in this embodiment. The CPU 114, acting as a computer, executes a computer program stored in memory, and each step of the flowchart in Figure 2 is performed sequentially. The processing details will be explained below based on the processing flow in Figure 2.

[0034] When the user starts shooting, the processing flow shown in Figure 2 begins, proceeding to step S201, and the action camera mode is activated. Specifically, in step S201, the user operates the control unit 115 while looking at the settings menu screen displayed on the display unit 113 to set the camera to action camera mode, which enables processing to correct large shakes such as those caused by walking or running.

[0035] When the user starts recording in action camera mode, an imaging step is performed in which the optical image formed by the optical system is captured by the imaging means.

[0036] In step S202, the irradiation control unit 123 receives irradiation instructions and current focal length information from the CPU 114, generates a control signal for the irradiation light source unit 122, and irradiates a line-shaped light (marker) corresponding to the lower end position of the shooting angle of view. That is, it irradiates a line-shaped marker as the projected shape of the irradiation light and controls its position.

[0037] Here, Figure 3 is a schematic diagram illustrating the principle of the method for controlling the irradiation angle of the illumination light. Also, Figure 4(A) is a schematic diagram illustrating the method for calculating the shooting angle of view, and Figure 4(B) is a schematic diagram illustrating the relationship between the orientation of the digital camera 100 and the lower edge of the shooting angle of view.

[0038] The following will explain how to control the direction of illumination of the light source and how to calculate the illumination angle, using Figures 3 and 4. First, we will explain how to control the direction of illumination of the light source.

[0039] Figure 3 shows an example configuration of a MEMS (Micro Electro Mechanical System) mirror that can change the optical path of a laser beam and perform scanning. A metal coil 301 is placed on a single-crystal silicon, a mirror 302 is formed inside the coil by MEMS processing, and a magnet 303 is placed below the mirror 302.

[0040] In the magnetic field 304 of the magnet, when an electric current 305 is passed through the coil 301 around the mirror 302, a Lorentz force 306 is generated according to Fleming's law, causing the mirror 302 to tilt.

[0041] In Figure 3, the mirror is driven in one axis (one dimension) to briefly explain the scanning principle of the laser light source using a MEMS mirror. However, by combining two springs formed by MEMS processing, it is possible to drive the mirror in two dimensions.

[0042] This allows for two-dimensional scanning of a single laser light source, enabling the formation of a line shape on the first axis and the modification of the irradiation angle on the second axis.

[0043] Next, we will explain how to calculate the angle at which the light source illuminates. As shown in Figure 4(A), if the vertical (y-direction) size of the image sensor 401 is d, the focal length from the principal point in the optical system 402 to the image sensor 401 is f, and the angle of view at that time is θ, then the angle of view θ can be calculated using the following equation 1.

[0044] θ = 2arctan(d / 2f) ... (Equation 1)

[0045] In Equation 1 above, let θx be the angle of view calculated based on the horizontal (x-direction) size of the image sensor 401, and let θy be the angle of view calculated based on the vertical (y-direction) size.

[0046] As shown in Figure 4(B), when the digital camera 100 is in any position or orientation, the irradiation angle range of the first axis of the laser light source described above is θx centered on the optical axis 404, and a line shape can be formed by rapidly scanning this angle range back and forth.

[0047] Furthermore, the illumination angle of the second axis is θy / 2 downward from the optical axis 404, making it possible to display a line-shaped illumination 405 corresponding to the lower end of the field of view as a marker.

[0048] Thus, in this embodiment, the imaging angle of view is calculated based on the size of the imaging means and the focal length of the optical system, and the irradiation information of the irradiation light from the irradiation means is controlled.

[0049] In the above explanation, the MEMS mirror method was used as an example for controlling the direction of light source illumination, but it is not limited to this method; other methods such as the galvanometer mirror method may also be used.

[0050] Furthermore, the above explanation briefly describes the principle of controlling the direction of light source illumination by assuming that the principal point position and the MEMS mirror position are the same within the optical system. However, in reality, their mounting positions are different, so control that takes this into account is necessary.

[0051] Figure 5 is a schematic diagram showing the method of displaying shooting information by visible light illumination according to Embodiment 1. User 501 holds the digital camera 100 without looking at the LCD display or electronic viewfinder in order to photograph the subject 502.

[0052] The illumination light unit 503, which includes one or more illumination light sources 122 and illumination control units 123, is positioned at the bottom of the digital camera 100 body or below the front of the lens, where interference with, for example, the shooting angle or the user's hand is unlikely. In other words, the illumination means is positioned at the bottom of the imaging device or below the front of the optical system.

[0053] Line 504, which serves as a line-shaped marker corresponding to the lower end of the shooting angle of view illuminated by the control method described above, and line 505, which serves as a line-shaped marker corresponding to the lower end of the cropped angle of view by the electronic image stabilization processing described later, are projected onto the ground or the like.

[0054] This results in a display that is visible to the user. Thus, in this embodiment, the illumination angle of the second axis described above is basically set to an angle that illuminates downwards from the horizontal. That is, the light emitted by the illumination means is directed downwards from the horizontal.

[0055] In step S202, the illumination light source unit 122 and the illumination control unit 123 illuminate a line corresponding to the lower end of the shooting angle of view, and the process proceeds to step S203. In step S203, the electronic image stabilization unit 121 performs electronic image stabilization to suppress blurring of the subject 502 and also performs angle of view cropping.

[0056] Figures 6(A) and 6(B) are illustrative diagrams showing an example of displaying shooting information by visible light illumination in Embodiment 1. In a scene where the camera is walking or running and taking a picture while following a subject 502 within the shooting angle 601 as shown in Figure 6(A), the image processing unit 104 performs image recognition and tracking processing of the subject 502 and outputs the subject area 603.

[0057] The electronic image stabilization unit 121 controls the subject area 603 to suppress blur based on the position information of the subject area 603 output in the image processing unit 104 and the movement vector detected based on feature points extracted from the captured angle of view image 601. In other words, it electronically suppresses blur by performing geometric deformation and angle of view cropping of the captured angle of view image 601.

[0058] The cropped recording angle of view obtained by the electronic image stabilization process is the angle of view cropped from the shooting angle of view 601 at a magnification corresponding to the focal length set in advance, and is, for example, an angle of view inside the shooting angle of view 601, as shown in 602 in Figure 6(A).

[0059] In step S203, the electronic image stabilization unit 121 performs electronic image stabilization processing, and the image processing unit 104 performs angle of view cropping processing. After this, the process proceeds to step S204.

[0060] In step S204, line illumination is performed at the lower end of the cropped field of view. Specifically, the illumination control unit 123 receives illumination instructions from the CPU 114 and information on the cropped field of view 602 obtained through electronic image stabilization processing, generates a control signal for the illumination light source unit 122, and illuminates the line 505 corresponding to the lower end of the cropped field of view 602.

[0061] The calculation of the illumination angle is the same as the line illumination corresponding to the lower end position of the shooting angle of view 601 in step S202, so the details are omitted. However, as mentioned above, the focal length must be considered in relation to the focal length of the shooting angle of view 601, taking into account the magnification corresponding to the pre-set focal length.

[0062] Furthermore, the illumination angle is adjusted according to the offset from the center of the shooting angle 601 to the center of the cropped angle 602. For example, in the horizontal direction, the angle of illumination that reciprocates to form a line shape is adjusted to an illumination angle that is asymmetrical with respect to the optical axis, while maintaining the angular range based on the focal length of the cropped angle 602.

[0063] Similarly, in the vertical direction, the angle is adjusted upward or downward by offsetting, using an angle of half the field of view downwards relative to the optical axis as a reference. In step S204, when the illumination light source unit 122 and the illumination control unit 123 illuminate a line corresponding to the lower end position of the cropped field of view, the process proceeds to step S205.

[0064] Here, steps S202 to S204 function as illumination steps that illuminate the front of the imaging means with illumination light and control the projection shape and position of the illumination light to notify (display) imaging information related to imaging by the imaging means.

[0065] In step S205, the CPU 114 determines whether the action camera mode has ended. That is, it determines whether the user has finished shooting in action camera mode. If the user turns off the power of the digital camera 100 or plays back the recorded data, the processing flow shown in Figure 2 ends.

[0066] In other words, the process of controlling the display of imaging information by visible light irradiation, as shown in Figure 2, is terminated. On the other hand, if imaging is to continue, the process returns to step S202 and steps S202 to S205 are repeated.

[0067] As described above, in this embodiment, a line-shaped marker is projected and displayed at a position corresponding to the lower end of the shooting angle of view and the cropped angle of view due to electronic image stabilization processing by visible light irradiation. This allows the user to take their eyes off the camera and grasp the surrounding situation, enabling them to simultaneously grasp information about framing and surrounding environmental information related to their own danger with the naked eye.

[0068] Regarding framing, the user can adjust the shooting direction based on the relative positional relationship between line 504, which corresponds to the lower edge of the shooting angle of view, and line 505, which corresponds to the lower edge of the cropped angle of view due to electronic image stabilization processing (604, 605, etc. in Figure 6(A)).

[0069] This allows the user to maintain a shooting angle where electronic image stabilization is effective, enabling video recording with suppressed blurring of the subject 502. Therefore, by displaying shooting information using visible light illumination during video shooting involving movement, it is possible to understand the surrounding situation while suppressing a decrease in video quality.

[0070] In the above embodiment 1, for example, an example was described in which, as information display using illuminated light, a line 504 corresponding to the lower end of the shooting angle of view and a line 505 corresponding to the lower end of the cropped angle of view due to electronic image stabilization processing are illuminated.

[0071] However, it is also possible to illuminate only one of the lines 504 corresponding to the lower end of the shooting angle of view and the line 505 corresponding to the lower end of the cropped angle of view due to the electronic image stabilization process. That is, the illumination means may illuminate a marker corresponding to the lower end of at least one of the imaging angle of view and the angle of view after electronic image stabilization. Furthermore, the illumination means may control the projection shape and position of the illuminated light based on at least one of the imaging angle of view by the imaging means and the angle of view after electronic image stabilization by the electronic image stabilization means.

[0072] Furthermore, as shown in Figure 6(B), for example, line 607 may be displayed above line 505, which corresponds to the lower end of the cropped field of view, at a distance of 608. Note that the interval 608 is the same as the interval 606 between the upper end of the shooting field of view and the upper end of the cropped field of view.

[0073] In other words, the illumination means may illuminate a marker that indicates the positional relationship between the upper end of the image field of view and the upper end of the field of view after electronic image stabilization. This allows the user to easily understand the distance 606 between the upper end of the shooting field of view and the upper end of the cropped field of view.

[0074] In this case, line 607, which corresponds to the upper edge of the cropped field of view, is illuminated in a position that does not appear within the cropped field of view, i.e., within the recording field of view. Alternatively, only one of the lines, line 504, which corresponds to the lower edge of the shooting field of view, or line 505, which corresponds to the lower edge of the cropped field of view due to electronic image stabilization processing, may be illuminated.

[0075] Furthermore, in the display of multiple pieces of information using visible light illumination, the color of each illuminated light may be the same or different. When there are multiple displays using illuminated light, for example, those illuminated based on the same photographic information may be illuminated with the same color light, while those illuminated based on different photographic information may be illuminated with different colored light. In other words, the illumination means may be configured to control the color of the illuminated light according to the photographic information.

[0076] <Embodiment 2> Next, an imaging system according to Embodiment 2 will be described with reference to Figures 7 to 11. Embodiment 2 describes an example in which imaging information is displayed using invisible light irradiation and a head-mounted display device. Specifically, the imaging system of Embodiment 2 has an imaging device according to Embodiment 1 and a head-mounted display device equipped with display means for imaging and displaying reflected invisible light.

[0077] The configuration of the digital camera 100 as an imaging device in Embodiment 2 is almost the same as in Embodiment 1, however, in Embodiment 2, the illumination light source unit 122 of the digital camera 100 emits invisible light, which is different from Embodiment 1.

[0078] Furthermore, the illumination light source unit 122 of the digital camera 100 may be configured to selectively emit invisible light and visible light. In that case, when the illumination light source unit 122 selectively emits visible light, the same operation as in Embodiment 1 can be performed.

[0079] Figure 7 is a functional block diagram showing an example configuration of the AR glasses 700, which are a head-mounted display device according to Embodiment 2. Figure 8 is a schematic external view of the AR glasses 700, which are a head-mounted display device according to Embodiment 2. Note that the AR (Augmented Reality) glasses 700 are a head-mounted display device.

[0080] In Figures 7 and 8, the imaging optical unit 701 includes multiple lens groups, including a focal lens. When taking a picture with the digital camera 100, the imaging optical unit 701 forms an optical image on the non-visible light image sensor 702. At the same time, exposure adjustment (aperture control, charge storage time control, etc.) is performed.

[0081] The invisible light image sensor 702 is an image sensor such as a CCD or CMOS sensor that converts the subject image received through an infrared filter into an electrical signal to create invisible light image data. The infrared filter is arranged corresponding to each pixel and has spectral characteristics such as having a peak in light transmittance in the infrared wavelength band of invisible light irradiated from the illumination light source 122, and substantially blocking the transmission of light in other wavelength bands.

[0082] The A / D conversion unit 703 converts the analog image signal from the invisible light image sensor 702 into a digital image signal. The converted image data is then input to the subsequent image processing unit 704.

[0083] The image processing unit 704 consists of various image processing circuits and buffer memory, and performs processes such as pixel correction, development, noise reduction, geometric transformation, and resizing on the image data converted by the A / D conversion unit 703. In addition, the image processing unit 704 performs processing to detect shooting information from the digital camera 100 using invisible light illumination from the developed and processed image.

[0084] Bus 714 is a system bus primarily used to transmit control signals from the CPU 712 and other components to each block, while bus 715 is a data bus primarily used to transfer image data.

[0085] The CPU 712, acting as a computer, controls the entire AR glasses 700, issuing operational instructions to each functional block and functioning as a control means to execute various control processes. The CPU 712 also performs calculations necessary for these various control processes.

[0086] Furthermore, the CPU 712 controls the image processing unit 704, data transfer unit 705, memory control unit 706, non-volatile memory control unit 708, display control unit 710, operation unit 713, non-visible light image sensor 702, etc., via the bus 714. The CPU 712 realizes each of the processes in this embodiment by executing the computer program recorded in the ROM 709.

[0087] The data transfer unit 705 is composed of multiple DMACs (Direct Memory Access Controllers) that perform data transfer.

[0088] The DRAM (memory) 707 is a memory that stores data, such as intermediate image data in image processing, constants for the operation of the CPU 712, and computer programs. The memory control unit 706 writes data to and reads data from the DRAM 707 in response to instructions from the CPU 712 or the data transfer unit 705.

[0089] The non-volatile memory control unit 708 writes and reads data from the ROM (non-volatile memory) 709 in response to instructions from the CPU 712. The ROM 709 is an electrically erasable and recordable memory, and EEPROM or similar is used. The ROM 709 stores constants for the operation of the CPU 712, computer programs, and the like.

[0090] Furthermore, the display unit 711, which serves as a display means, consists of an optically transmitted display and is controlled by the display control unit 710. It displays images in real time based on the shooting information obtained by invisible light irradiation detected by the image processing unit 704, as well as various setting screens.

[0091] The information may be displayed using a GUI (Graphical User Interface). The operating unit 713, which is the means of operation, is a touchpad or the like that can be operated by the user and is used for various operations.

[0092] 716 is a communication unit and functions as a means of communication between the AR glasses 700 and the imaging device, such as the digital camera 100. The communication by the communication unit 716 can be wireless or wired. The communication unit 716 receives synchronization and control signals from the digital camera 100, and the CPU 712 in the AR glasses 700 performs various operations in conjunction with (synchronized with) the operation of the digital camera 100, as shown in the flowchart of Figure 9 described later.

[0093] As shown in Figure 8, the display unit 711, which is an optically transmitted display, is provided in a part of the glass portion of the AR glasses 700. Therefore, a user wearing the AR glasses 700 can view the images displayed on the display unit 711 while also being able to see the surrounding environment of the AR glasses 700.

[0094] Figure 9 is a flowchart showing an example of the processing of displaying imaging information by invisible light irradiation in the imaging system according to Embodiment 2. Figure 9 shows the processing flow in the processing of controlling the display of imaging information by invisible light irradiation of a digital camera 100 as an imaging device and AR glasses 700 as a head-mounted display device.

[0095] Furthermore, the CPUs 114 and 712 of the digital camera 100 and AR glasses 700, acting as computers, execute the computer programs stored in memory, thereby sequentially performing the operations at each step of the flowchart in Figure 9. The processing details will be explained below based on the processing flow in Figure 9.

[0096] When the user puts the AR glasses 700 on their head and the digital camera 100 starts taking pictures, the processing flow shown in Figure 9 starts, and the process proceeds to step S901.

[0097] In step S901, the action camera mode is started. Specifically, the user operates the control unit 115 while looking at the settings menu screen displayed on the display unit 113 of the digital camera 100 to set it to action camera mode, which enables processing to correct large shakes such as when shooting while walking or running. When the user starts recording in action camera mode, the process proceeds to step S902.

[0098] In step S902, the illumination control unit 123 of the digital camera 100 receives illumination instructions and current focal length information from the CPU 114, generates a control signal for the illumination light source unit 122, and illuminates the line corresponding to the lower end of the shooting angle of view with invisible light.

[0099] Figure 10 is a schematic diagram showing the method of displaying shooting information by invisible light illumination according to Embodiment 2. User 501 wears AR glasses 700 and holds the digital camera 100 to photograph the subject 502 without looking at the LCD display or electronic viewfinder of the digital camera 100.

[0100] The illumination light unit 503, which includes one or more illumination light sources 122 and illumination control units 123, is positioned at the bottom of the digital camera 100 body or below the lens tip, where it is less likely to interfere with, for example, the shooting angle or the user's hand.

[0101] Line 1001, which serves as a line-shaped marker corresponding to the lower end of the shooting angle of view illuminated with invisible light, and line 1002, which serves as a line-shaped marker corresponding to the lower end of the cropped angle of view due to the electronic image stabilization process described later, are illuminated and reflected off the ground or other surface. This makes them detectable by the invisible light image sensor.

[0102] In step S902, the illumination light source unit 122 and illumination control unit 123 of the digital camera 100 illuminate a line corresponding to the lower end position of the shooting angle of view, and then the process proceeds to step S203. In step S203, similar to Embodiment 1, the electronic image stabilization unit 121 performs electronic image stabilization to suppress blurring of the subject 502 and also performs angle of view cropping.

[0103] Next, in step S903, the line illumination of the subject position to be electronically corrected is performed. That is, the illumination control unit 123 of the digital camera 100 receives illumination instructions from the CPU 114 and information on the subject area 603 to be corrected, and generates a control signal for the illumination light source unit 122.

[0104] Then, as shown in Figure 11(A), the line 1101 corresponding to the lower end position of the subject area 603 is illuminated with invisible light. That is, the subject area is obtained from the image obtained from the imaging means, and the projection shape and position of the illumination light are controlled based on the subject area.

[0105] Figures 11(A) and (B) are illustrative diagrams showing an example of displaying shooting information by invisible light irradiation according to Embodiment 2. In step S903, when the line 1101 corresponding to the lower end position of the subject area 603 is illuminated by the illumination light source unit 122 and illumination control unit 123 of the digital camera 100, the process proceeds to step S904.

[0106] In step S904, a line is illuminated at the lower end of the cropped field of view. Specifically, the illumination control unit 123 of the digital camera 100 receives illumination instructions from the CPU 114 and information on the cropped field of view 602 obtained through electronic image stabilization processing, and generates a control signal for the illumination light source unit 122. Then, it illuminates the line 1002 corresponding to the lower end of the cropped field of view 602 with invisible light.

[0107] Next, in step S905, a non-visible light image is acquired by the head-mounted display device. That is, the non-visible light image sensor 702 of the AR glasses 700 acquires a non-visible light image in the direction of the user's line of sight, including the non-visible light emitted from the illumination light source unit 122 of the digital camera 100. This image is then output to the image processing unit 704 via the A / D conversion unit 703. Once the non-visible light image in the direction of the user's line of sight is acquired in step S905, the process proceeds to step S906.

[0108] In step S906, the illuminated light region is detected from the invisible light image. That is, the image processing unit 704 of the AR glasses 700 extracts the region of invisible light illuminated by the illumination light source unit 122 of the digital camera 100 from the acquired invisible light image.

[0109] Extraction of the invisible light region is performed, for example, by setting the spectral characteristics of the infrared filter of the invisible light image sensor 702 to have the highest transmittance in the wavelength band of the illumination light source and to substantially block light in other wavelength bands, and then extracting the region in which the pixel value is above a predetermined value. Once the invisible light illumination region is extracted in step S906, the process proceeds to step S907.

[0110] In step S907, the system displays the image information (e.g., a line-shaped marker) based on the illuminated light area. Specifically, the CPU 712 of the AR glasses 700 displays the image information at the corresponding location in the user's field of view, based on the focal length of the imaging optical unit 701, the size of the invisible light image sensor 702, and the positional relationship between the invisible light image sensor 702 and the display unit 711.

[0111] Specifically, the CPU 712 issues a control command to the display control unit 710 to display the image information (for example, a line-shaped marker) based on invisible light illumination at the corresponding location in the user's field of view.

[0112] The display control unit 710, in accordance with control commands from the CPU 712, displays invisible light capture information (e.g., line-shaped markers) emitted from the digital camera 100 on the display unit 711. As a result, the user can obtain a view like that shown in Figure 11(A) through the optically transmitted display of the AR glasses 700.

[0113] Furthermore, line 1001 corresponding to the lower end of the shooting angle of view, line 1002 corresponding to the lower end of the cropped angle of view due to electronic image stabilization processing, and line 1101 corresponding to the lower end of the subject area to be photographed can be seen. In step S907, when the display unit 711 of the AR glasses 700 displays the shooting information (e.g., line-shaped markers) extracted by invisible light illumination, the process proceeds to step S908.

[0114] In step S908, the CPU 114 of the digital camera 100 determines whether the user has finished shooting in action camera mode. If the user turns off the power of the digital camera 100 or plays back the recorded data, the shooting information display control processing flow using invisible light illumination, as shown in Figure 9, is terminated. At this time, the power of the AR glasses may also be turned off in conjunction with this.

[0115] If the result in step S908 is NO, the process returns to step S902 and steps S902 to S908 are repeated.

[0116] In this embodiment, invisible light is used to illuminate the position corresponding to the lower end of the shooting angle and the cropped angle due to electronic image stabilization processing, as well as the position of the subject recognized as the target of shooting, and these are displayed on an optically transmitted display. Therefore, the user can simultaneously grasp information about the surrounding environment that may pose a risk to their safety, along with information about the framing.

[0117] Regarding framing, similar to Embodiment 1, the user can adjust the shooting direction based on the relative positional relationship between the shooting angle of view and the cropped angle of view (604 and 605 in Figure 11(A)). This allows the user to maintain a shooting angle of view in which the electronic image stabilization processing is effective, enabling video recording with suppressed blur of the subject 502 being filmed.

[0118] Furthermore, by using invisible light for illumination, it becomes possible to illuminate invisible light markers, such as line-shaped markers, within the recording field of view for the purpose of capturing information. This allows the AR Glasses 700 to provide users with more information that is useful for shooting.

[0119] In other words, by using invisible light illumination to provide information about the shooting process during video recording while moving, and displaying it on an optically transmitted display, it is possible to understand the surrounding situation while suppressing a decrease in video quality.

[0120] In Embodiment 2, for example, an example was described in which line 1001 corresponding to the lower end of the shooting angle of view and line 1002 corresponding to the lower end of the cropped angle of view due to electronic image stabilization processing are illuminated as information display by non-visible light illumination, but the invention is not limited to this.

[0121] For example, as shown in Figure 11(B), line 1102 may be displayed above line 1002, which corresponds to the lower end of the cropped field of view, at a distance of 608. Note that the distance of 608 is the same as the distance 606 between the upper end of the shooting field of view and the upper end of the cropped field of view. This allows the user to easily understand the distance 606 between the upper end of the shooting field of view and the upper end of the cropped field of view. Alternatively, only one of the lines, line 1001, which corresponds to the lower end of the shooting field of view, or line 1002, which corresponds to the lower end of the cropped field of view due to electronic image stabilization processing, may be illuminated.

[0122] Furthermore, while an example of displaying information using invisible light illumination, such as displaying lines corresponding to the lower edge of each field of view, has been described, the method is not limited to this. The lines corresponding to the left, right, and upper edges, shown by the dashed lines of the shooting field of view 601 and cropped field of view 602 in Figure 11(A), may be estimated from the line information corresponding to the lower edge and displayed on an optically transmitted display.

[0123] Furthermore, in information display using invisible light illumination, the display color of each illuminated light on an optically transmitted display may be colorless or the same color. Alternatively, if there are multiple invisible light illumination areas extracted by the AR glasses 700, areas within a predetermined coordinate distance range on the image may be displayed in the same color, while areas outside the predetermined coordinate distance may be displayed in different colors. In other words, the display means may control the display color of the reflected invisible light according to the captured information.

[0124] <Embodiment 3> Next, an imaging device according to Embodiment 3 will be described with reference to Figures 12 and 13. Embodiment 3 describes an example in which visible light irradiation is performed outside the exposure period (charge accumulation period) of the image sensor of the digital camera 100, and imaging information (for example, a line-shaped marker) is displayed. That is, in Embodiment 3, the irradiation means irradiates light outside the exposure period by the imaging means.

[0125] The configuration of the digital camera used as the imaging device in this embodiment is the same as in Embodiment 1, and some of the processing for displaying and controlling the image information using visible light illumination is also the same as in Embodiment 1. Only the differences will be described, and the descriptions of the identical parts will be omitted.

[0126] Figure 12 is a flowchart showing an example of the imaging information display process using visible light irradiation in Embodiment 3. The CPU 114, acting as a computer, executes a computer program stored in memory, sequentially performing each step in the flowchart of Figure 12. The processing details will be explained below based on the processing flow shown in Figure 12.

[0127] In step S201, action camera mode is activated. That is, the user starts recording in action camera mode.

[0128] Next, in step S1201, the image sensor acquires an image (exposure). That is, the image sensor 102 receives a control instruction from the CPU 114 and performs exposure (charge accumulation operation) for a predetermined exposure period (charge accumulation period) to acquire an image within the shooting angle of view. Once the exposure of the image sensor 102 is complete, the process proceeds to step S1202.

[0129] In step S1202, line illumination is performed at the lower end of the imaging field of view. That is, the illumination control unit 123 receives the illumination instruction from the CPU 114 and the current focal length information, as well as the exposure period information of the image sensor 102, and generates a control signal for the illumination light source unit 122 to perform line illumination.

[0130] Figure 13 is a diagram showing an example of the control timing for displaying imaging information by visible light irradiation in Embodiment 3, and is a diagram showing the timing relationship between the exposure period (charge accumulation period) of the image sensor 102 and the irradiation period by the irradiation control unit 123 and the irradiation light source unit 122.

[0131] The upper panel shows the exposure period of the image sensor 102, and the lower panel shows the illumination period by the illumination control unit 123 and the illumination light source unit 122. The horizontal axis is the time axis, and the Nth and N+1th frames, which are part of the image capture, are extracted.

[0132] Between the exposure periods of each frame of the image sensor 102, there is a blanking period 1302 during which no exposure occurs, and the imaging signal is read out from the image sensor 102 during this blanking period.

[0133] The irradiation control unit 123 generates a control signal to the irradiation light source unit 122 so that irradiation does not occur during the exposure period 1301 of the image sensor 102. It also generates a control signal to the irradiation light source unit 122 so that irradiation occurs during the blanking period 1202 of the image sensor 102.

[0134] In step S1202 of Figure 12, the illumination light source unit 122 illuminates a line corresponding to the lower end of the shooting angle of view, in accordance with the control signal from the illumination control unit 123. Then, in step S203, similar to Embodiment 1, the electronic image stabilization unit 121 performs electronic image stabilization to suppress blurring of the subject 502 and also performs angle of view cropping.

[0135] Next, in step S1203, line illumination is performed at the lower end of the cropped field of view. Specifically, the illumination control unit 123 receives illumination instructions from the CPU 114 and cropped field of view information from the electronic image stabilization process, as well as exposure period information from the image sensor 102, and generates a control signal for the illumination light source unit 122.

[0136] Furthermore, in step S1203, similar to step S1202, the irradiation control unit 123 generates a control signal for the irradiation light source unit 122 so that irradiation does not occur during the period 1301 corresponding to the exposure period of the image sensor 102. Then, it generates a control signal for the irradiation light source unit 122 so that irradiation occurs during the blank period 1202 of the image sensor 102, and the irradiation light source unit 122 irradiates the line corresponding to the lower end position of the cropped field of view according to the control signal from the irradiation control unit 123.

[0137] Next, in step S205, the CPU 114 determines whether the user has finished shooting in action camera mode. That is, if the user turns off the power of the digital camera 100 or plays back the recorded data, the visible light illumination shooting information display control processing flow shown in Figure 12 is terminated. If shooting is to continue, the process returns to step S1201 and repeats the processing from steps S1201 to S205.

[0138] As explained above, by displaying the shooting information using visible light illumination outside the exposure period of the image sensor 102, it becomes possible to illuminate within the video recording field of view, for example, the marker for the position of the subject recognized as the shooting target as shown in Embodiment 2.

[0139] This allows users to simultaneously capture information about their surrounding environment that could pose a risk to themselves, as well as information about markers and other captured data, within their field of view while recording. This enables them to understand their surroundings while minimizing any degradation in video quality.

[0140] <Embodiment 4> Next, an imaging device according to Embodiment 4 will be described with reference to Figures 14 and 15. Embodiment 4 describes an example in which additional information related to the shooting other than the field of view is displayed by visible light illumination.

[0141] The configuration of the digital camera used as the imaging device in this embodiment is the same as in Embodiment 1, and part of the processing for displaying shooting information using visible light illumination is also the same as in Embodiment 1. Only the different parts will be described, and the description of the identical parts will be omitted.

[0142] Figure 14 is a flowchart showing an example of the display process of imaging information and imaging-related information using visible light irradiation in Embodiment 4. The CPU 114, acting as a computer, executes a computer program stored in memory, sequentially performing each step in the flowchart of Figure 14. The processing details will be explained below based on the processing flow of Figure 14. Steps S201 to S205 are the same as in Embodiment 1, so their explanation will be omitted.

[0143] In step S1401 of Figure 14, a simplified pattern irradiation of shooting-related information is performed. Specifically, the irradiation control unit 123 receives irradiation instructions from the CPU 114 and shooting-related information such as the status of the digital camera 100, generates a control signal for the irradiation light source unit 122, and irradiates a simplified pattern that has been previously linked to (associated with) the shooting-related information.

[0144] In this embodiment, the shooting-related information refers to information such as the remaining battery level and remaining recording time of the digital camera 100. When these fall below a certain level, illumination is performed to display a warning to the user as shooting-related information. Thus, in this embodiment, the illumination means emits illumination light according to an illumination pattern that is pre-associated with the shooting-related information.

[0145] Figure 15 is a schematic diagram showing the display method of shooting information and shooting-related information by visible light irradiation in Embodiment 4. Separately from the information 504, 504 which is related to the field of view, a simplified pattern 1501 of shooting-related information is irradiated from the irradiation light unit 503.

[0146] Furthermore, since the ground or other surface onto which the projected light is directed is not necessarily flat, even if the projection uses the same shapes and characters as when displaying information such as battery level on the digital camera 100's display, visibility may be poor.

[0147] Therefore, in this embodiment, the simplified pattern 1501 that is pre-linked (associated) with the shooting-related information is used as a marker with less information than the content displayed on the digital camera 100's display.

[0148] This makes it easier for the user to recognize the image even when projection conditions are relatively poor, such as when the projection surface has irregularities. In step S1401, when the illumination light source unit 122 and the illumination control unit 123 illuminate a simplified pattern of imaging-related information, the process proceeds to step S205.

[0149] In step S205 of Figure 14, the CPU 114 determines whether the user has finished shooting in action camera mode. If the user turns off the power of the digital camera 100 or plays back the recorded data, the visible light illumination shooting information display control processing flow in Figure 14 is terminated. On the other hand, if shooting continues, the process returns to step S202 and repeats the processing in steps S202 to S205 of Figure 14.

[0150] As explained above, additional shooting-related information is displayed for shooting beyond the field of view using visible light illumination. Therefore, users can see not only information about the surrounding environment that may pose a risk to themselves during shooting, but also shooting information such as markers, and further shooting-related information, allowing them to understand their surroundings while suppressing a decrease in video quality.

[0151] Furthermore, while we have described an example of displaying additional warnings to the user when the battery level or remaining recording time falls below a certain level, this is not the only example. For instance, the battery level or remaining recording time could be displayed in multiple levels, or the various settings and menus of the Digital Camera 100 could be displayed.

[0152] <Embodiment 5> Next, an imaging device according to Embodiment 5 will be described with reference to Figures 16 to 18. Embodiment 5 describes an example in which imaging information such as the field of view and imaging-related information are displayed using invisible light irradiation and a head-mounted display device.

[0153] The configuration of the digital camera as an imaging device and the AR glasses as a head-mounted display device in this embodiment are the same as in Embodiment 2, and the light emitted by the illumination unit of the digital camera is invisible light. Furthermore, some of the processing for displaying the shooting information using visible light illumination is the same as in Embodiment 2, so only the different parts will be described, and the description of the identical parts will be omitted.

[0154] Figure 16 is a flowchart illustrating an example of the display process for shooting information and related information using invisible light irradiation in Embodiment 5. The CPU 114 of the digital camera 100 and the CPU 712 of the AR glasses execute computer programs stored in their respective memories, sequentially performing each step in the flowchart of Figure 16. The processing details will be explained below based on the processing flow of Figure 16. Steps S901 to S906 and S908 are the same as in Embodiment 2, so their explanation will be omitted.

[0155] In step S1601 of Figure 16, a simplified pattern illumination of shooting-related information is performed. Specifically, the illumination control unit 123 generates a control signal for the illumination light source unit 122 based on the illumination instruction from the CPU 114 and shooting-related information such as the status of the digital camera 100, and illuminates a simplified pattern linked to the shooting-related information in advance with invisible light.

[0156] Figure 17 is a schematic diagram showing the display method of imaging information and imaging-related information using invisible light irradiation in Embodiment 5. Separately from imaging information related to the field of view (e.g., line-shaped markers), a simplified pattern 1701 of imaging-related information is irradiated with invisible light from the illumination light unit 503, and becomes detectable by the invisible light image sensor of the AR glasses 700 when it hits the ground or the like and reflects.

[0157] In step S1601, when the illumination light source unit 122 and the illumination control unit 123 illuminate a simplified pattern of imaging-related information, the process proceeds to step S905. Steps S905 and S906 are the same as in Embodiment 2.

[0158] In step S1602 of Figure 16, the system displays the image information (e.g., line-shaped markers) based on the illuminated light area and the image-related information. Specifically, the CPU 712 of the AR glasses 700 displays the extracted image information (e.g., line-shaped markers) based on invisible light illumination, and also uses image recognition to reconstruct the original image-related information from a simplified pattern of the image-related information.

[0159] In other words, in this embodiment, the irradiation means irradiates invisible light with an irradiation pattern pre-associated with the shooting-related information, and the display means restores and displays the shooting-related information pre-associated with the irradiation pattern. The display control unit 710 then issues a control command to display the restored shooting-related information on the display unit 711 as a GUI that is easy to recognize visually and intuitively.

[0160] Figure 18 is an image diagram showing an example of displaying shooting-related information using invisible light illumination in Embodiment 5. For example, as shown in Figure 18, if the original shooting-related information to which the circular simple pattern 1701 is pre-associated is a low battery warning, it is converted to a battery level mark like 1801 in Figure 19.

[0161] Then, the display control unit 710, in accordance with the control commands from the CPU 712, displays a battery level mark like 1801 on the display unit 711.

[0162] Next, in step S908, the CPU 114 of the digital camera 100 determines whether the user has finished shooting in action camera mode. If the user turns off the power of the digital camera 100 or plays back the recorded data, the shooting information display control processing flow by invisible light illumination shown in Figure 16 is terminated.

[0163] In addition, the power to the AR glasses may be turned off in conjunction with this. On the other hand, if step S908 is determined to be NO and shooting continues, the process returns to step S902 and steps S902 to S908 are repeated.

[0164] As described above, in Embodiment 5, by using invisible light irradiation and a head-mounted display device to display additional shooting-related information in addition to shooting information such as the field of view, the user can visually and intuitively recognize useful information.

[0165] Furthermore, users can simultaneously capture information about their surrounding environment that may pose a risk, along with shooting information (such as line-shaped markers) and shooting-related information within their field of view during shooting. This allows them to understand their surroundings while simultaneously minimizing the degradation of video quality.

[0166] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible in accordance with the spirit of the present invention, and these are not excluded from the scope of the present invention. Furthermore, some of the above embodiments may be combined as appropriate.

[0167] Furthermore, the present invention includes, for example, a system that implements the functions of the above embodiment using at least one processor such as a CPU, memory, and circuitry (e.g., an ASIC). Alternatively, multiple processors may be used for distributed processing.

[0168] Furthermore, in order to implement some or all of the control in the above embodiment, a computer program that implements the functions of the above embodiment may be supplied to the imaging device, etc., via a network or various storage media.

[0169] The computer (or CPU or MPU, etc.) in the imaging device may read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention. The present invention includes the following combinations.

[0170] (Configuration 1) An imaging device characterized by comprising: an imaging means for capturing an optical image formed by an optical system; and an illumination means for irradiating illumination light in front of the imaging means and for notifying imaging information related to the imaging means by controlling the projection shape and position of the illumination light.

[0171] (Configuration 2) The imaging apparatus according to Configuration 1, characterized in that the irradiation means irradiates with visible light.

[0172] (Configuration 3) The imaging apparatus according to Configuration 1 or 2, characterized in that the irradiation means irradiates the irradiation light outside the exposure period by the imaging means.

[0173] (Configuration 4) An imaging device according to any one of Configurations 1 to 3, characterized in that it calculates the imaging angle of view based on the size of the imaging means and the focal length of the optical system, and controls the irradiation information of the irradiation light by the irradiation means.

[0174] (Configuration 5) The imaging device according to any one of Configurations 1 to 4, characterized in that the irradiation light from the irradiation means is irradiated downward from the horizontal.

[0175] (Configuration 6) The imaging device according to any one of Configurations 1 to 5, characterized in that the irradiation means is located at the lower part of the imaging device or at the lower part of the tip of the optical system.

[0176] (Configuration 7) An imaging device according to any one of Configurations 1 to 6, wherein the imaging device has an electronic image correction means that performs electronic image correction on an image captured by the imaging means, and the illumination means controls the projection shape and position of the illumination light based on at least one of the imaging angle of view by the imaging means and the angle of view after electronic image correction by the electronic image correction means.

[0177] (Configuration 8) The imaging apparatus according to Configuration 7, characterized in that the illumination means illuminates a marker corresponding to at least one of the lower ends of the imaging angle of view and the angle of view after electronic image correction.

[0178] (Configuration 9) The imaging apparatus according to Configuration 7 or 8, characterized in that the illumination means illuminates a marker indicating the positional relationship between the upper end of the imaging angle of view and the upper end of the angle of view after electronic image correction.

[0179] (Configuration 10) An imaging device according to any one of Configurations 1 to 9, characterized in that it acquires a subject area from an image obtained from the imaging means and controls the projection shape and position of the irradiated light based on the subject area.

[0180] (Configuration 11) The imaging device according to any one of Configurations 1 to 10, characterized in that the irradiation means controls the color of the irradiation light according to the imaging information.

[0181] (Configuration 12) The imaging apparatus according to any one of Configurations 1 to 11, characterized in that the irradiation means irradiates the irradiation light with an irradiation pattern that is pre-associated with the imaging-related information relating to the imaging.

[0182] (Configuration 13) The imaging apparatus according to any one of Configurations 1 to 12, characterized in that the irradiation means irradiates with non-visible light.

[0183] (Configuration 14) An imaging system characterized by comprising the imaging device described in Configuration 13 and a head-mounted display device equipped with a display means for imaging and displaying the reflected light of the non-visible light.

[0184] (Configuration 15) The imaging system according to Configuration 14, characterized in that the display means is an optically transmitted display.

[0185] (Configuration 16) The imaging system according to Configuration 14 or 15, characterized in that the display means controls the display color of the reflected light of the non-visible light according to the shooting information.

[0186] (Configuration 17) The imaging system according to any one of Configurations 14 to 16, characterized in that the illumination means irradiates the invisible light of an illumination pattern that is pre-associated with the imaging-related information relating to the imaging, and the display means restores and displays the imaging-related information that is pre-associated with the illumination pattern.

[0187] (Method) An imaging method characterized by comprising: an imaging step of imaging an optical image formed by an optical system using an imaging means; and an illumination step of irradiating an illumination light in front of the imaging means and controlling the projection shape and position of the illumination light to provide imaging information related to imaging by the imaging means.

[0188] (Program) A computer program for controlling each means of the imaging device described in any one of configurations 1 to 13 by computer. [Explanation of Symbols]

[0189] 100: Digital Camera 122: Irradiation light source section 503: Irradiation light unit 504, 505: Irradiation light 700: AR Glasses 711: Display section

Claims

1. An imaging means for capturing an optical image formed by an optical system, An illumination means that emits illumination light in front of the imaging means and controls the projection shape and position of the illumination light to notify imaging information related to imaging by the imaging means, An imaging device characterized by having the following features.

2. The imaging apparatus according to claim 1, characterized in that the irradiation means irradiates with visible light.

3. The imaging apparatus according to claim 1, characterized in that the irradiation means irradiates with the irradiation light outside the exposure period by the imaging means.

4. The imaging apparatus according to claim 1, characterized in that it calculates the imaging angle of view based on the size of the imaging means and the focal length of the optical system, and controls the irradiation information of the irradiation light by the irradiation means.

5. The imaging apparatus according to claim 1, characterized in that the illumination light from the illumination means is directed downward from the horizontal.

6. The imaging apparatus according to claim 1, characterized in that the irradiation means is located at the lower part of the imaging apparatus or at the lower part of the tip of the optical system.

7. The system includes an electronic image stabilization means that performs electronic image stabilization on the image captured by the aforementioned imaging means. The imaging apparatus according to claim 1, characterized in that the illumination means controls the projection shape and position of the illumination light based on at least one of the imaging angle of view by the imaging means and the angle of view after electronic image correction by the electronic image correction means.

8. The imaging apparatus according to claim 7, characterized in that the illumination means illuminates a marker corresponding to at least one of the lower ends of the imaging angle of view and the angle of view after electronic image correction.

9. The imaging apparatus according to claim 7, characterized in that the illumination means illuminates a marker indicating the positional relationship between the upper end of the imaging angle of view and the upper end of the angle of view after electronic image correction.

10. The imaging device according to claim 1, characterized in that it acquires a subject area from an image obtained from the imaging means and controls the projection shape and position of the irradiated light based on the subject area.

11. The imaging device according to claim 1, characterized in that the irradiation means controls the color of the irradiation light according to the imaging information.

12. The imaging apparatus according to claim 1, characterized in that the irradiation means irradiates with irradiation light of an irradiation pattern that is pre-associated with the imaging-related information relating to the imaging.

13. The imaging apparatus according to claim 1, characterized in that the irradiation means irradiates with non-visible light.

14. The imaging device according to claim 13, An imaging system characterized by having a head-mounted display device equipped with a display means for capturing and displaying reflected light of the aforementioned invisible light.

15. The imaging system according to claim 14, characterized in that the display means is an optically transmitted display.

16. The imaging system according to claim 14, characterized in that the display means controls the display color of the reflected light of the non-visible light according to the shooting information.

17. The irradiation means irradiates the invisible light with an irradiation pattern that is pre-associated with the imaging-related information relating to the imaging, The imaging system according to claim 14, characterized in that the display means restores and displays imaging-related information that has been pre-associated with the irradiation pattern.

18. An imaging step in which an optical image formed by an optical system is captured by an imaging means, An illumination step in which illumination light is shone in front of the imaging means, and the projection shape and position of the illumination light are controlled to provide information about the imaging means taking photographs. An imaging method characterized by having the following features.

19. A computer program for controlling each means of an imaging apparatus according to any one of claims 1 to 13 by computer.

Images