Imaging apparatus, information processing apparatus, control method, program, and storage medium

Abstract

To provide an imaging apparatus, a communication system, a control method, a program, and a storage medium that can accurately perform pattern recognition without reducing customer satisfaction and can perform imaging control according to a photographic scene based on a result of the pattern recognition.SOLUTION: An imaging apparatus 100 comprises: an image pick-up device 101 including pixels that receive visible light from light beams from a subject and output a visible light image, and pixels that receive non-visible light from the light beams from the subject and output a non-visible light image; system control means 102 that switches imaging control in picking up the visible light image; and pattern recognition means 103 that performs pattern recognition for the non-visible light image. When a specific pattern is detected from the non-visible light image through the pattern recognition performed by the pattern recognition means 103, the system control means 102 switches the imaging control from a strobe light emission permission mode to a strobe light emission inhibition mode.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device, an information processing device, a control method, a program, and a storage medium, and more particularly to an imaging device, an information processing device, a control method, a program, and a storage medium that switch imaging control depending on the shooting scene. [Background technology]

[0002] In certain photography situations, there are actions that are prohibited when taking photographs, i.e., photography taboos. For example, in sports such as golf and tennis, sound recording is often prohibited. Furthermore, flash photography is often prohibited in art appreciation venues, such as museums. Traditionally, to prevent taboo photography behaviors, no-photography signs have been placed next to exhibits or announcements made in advance within the venue that photography is prohibited. However, these methods cannot prevent photographers from accidentally engaging in taboo photography behaviors, which can result in photographers being forced to leave or having their licenses revoked, potentially resulting in the loss of photographic opportunities. Furthermore, in recent years, expensive cameras have been rented out at museums and sporting events, such as the Olympics, to ensure photographers do not miss crucial moments or memories. Therefore, to prevent photographers from accidentally engaging in taboo photography behaviors, it is necessary for the camera, not the photographer, to determine which actions constitute taboo photography.

[0003] Furthermore, one of the obstacles to the widespread adoption of wearable devices such as smart glasses is the problem of privacy violations caused by photography and video recording anywhere. To avoid this problem, it is necessary to implement image capture control so that wearable devices cannot automatically photograph or record under certain conditions.

[0004] In order to prevent taboo photography using devices such as cameras and wearable devices and photography and recording that violate privacy, it is desirable for the device to appropriately control photography based on the subject recognition results. For example, Patent Document 1 discloses a camera that prohibits flash emission when fireworks or a specific person are detected as the subject. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-166077 Summary of the Invention [Problem to be solved by the invention]

[0006] However, conventional methods such as placing no-photography signs near exhibits or making announcements within the venue not only fail to prevent the unintentional photography taboo mentioned above, but also have the potential to lower customer satisfaction. For example, due to space limitations, no-photography signs may only be placed in places that obstruct the view, or the announcements may disturb customers who wish to enjoy art in peace.

[0007] Furthermore, in Patent Document 1, multiple patterns are stored for recognizing fireworks or a specific person as a subject, but there are cases where it is difficult to recognize the pattern of fireworks or a specific person using the stored patterns. For example, this occurs when the subject changes to a shape that is different from the stored pattern, such as when the shape of the fireworks is distorted by the wind or when a specific person is facing away. In addition, it is necessary to store a large number of patterns for each scene, which leads to increased costs.

[0008] Therefore, the present invention provides an imaging device, an information processing device, a control method, a program, and a storage medium that can perform pattern recognition with high accuracy without reducing customer satisfaction, and that can control imaging according to the shooting scene based on the pattern recognition results. [Means for solving the problem]

[0009] In order to solve the above problem, the imaging device according to claim 1 of the present invention comprises an imaging element having a photoelectric conversion unit including a first pixel that receives visible light from a light beam from a subject and outputs a visible light image, and a second pixel that receives non-visible light from the light beam from the subject and outputs a non-visible light image, a control means that switches imaging control when capturing the visible light image, and a pattern recognition means that performs pattern recognition on the non-visible light image, wherein the control means switches the imaging control from the first imaging control to the second imaging control when a specific pattern is detected from the non-visible light image by pattern recognition by the pattern recognition means.

[0010] In order to solve the above problem, the information processing device according to claim 24 of the present invention is an information processing device that links a plurality of imaging devices, and is characterized by comprising: a pattern recognition means that, when receiving from one of the plurality of imaging devices an invisible light image generated from invisible light in a light beam from a subject, performs pattern recognition on the received invisible light image; and an instruction transmission means that, when a specific pattern is detected from the received invisible light image through pattern recognition by the pattern recognition means, transmits to the plurality of imaging devices a switch instruction to switch the imaging control of each of the plurality of imaging devices from a first imaging control to a second imaging control. [Effects of the Invention]

[0011] According to the present invention, pattern recognition can be performed with high accuracy without reducing customer satisfaction, and imaging control can be performed according to the shooting scene based on the pattern recognition results. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a hardware configuration of an imaging apparatus according to a first embodiment. [Figure 2] FIG. 2 is a diagram schematically illustrating a part of the configuration of an imaging element. [Figure 3] 4 is a flowchart of a photographing process executed in the imaging device according to the first embodiment. [Figure 4] 10A and 10B are diagrams illustrating examples of specific patterns detected from non-visible light images. [Figure 5] 1 is a diagram illustrating a photographic scene including a subject on which a sticker with a specific pattern drawn in invisible light ink is attached. [Figure 6] FIG. 10 is a diagram showing a live view screen according to the first embodiment when a specific pattern is detected from a non-visible light image. [Figure 7] FIG. 10 is a diagram showing a menu screen displayed when the user turns off the reading of an invisible light signal according to the first embodiment. [Figure 8] FIG. 10 is a diagram showing a movie theater screen as a subject, which is captured by live view photography using an imaging device according to a second embodiment. [Figure 9] 9 is a diagram illustrating an example of the timing at which the projection device in FIG. 8 projects a specific pattern onto a screen using invisible light. [Figure 10] 10 is a flowchart of a photographing process executed in an imaging device according to a second embodiment. [Figure 11] FIG. 10 is a block diagram showing the hardware configuration of an imaging apparatus according to a third embodiment. [Figure 12] FIG. 10 is a diagram illustrating a configuration of a communication system according to a fourth embodiment. [Figure 13] FIG. 14 is a block diagram showing the hardware configuration of the imaging device in FIG. [Figure 14] FIG. 10 is a sequence diagram showing the flow of imaging processing according to the fourth embodiment, which is executed in a server and a plurality of imaging devices. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each drawing, the same reference numerals are used to refer to the same components, and duplicated explanations will be omitted. Note that the embodiments described below are merely examples, and the present invention is not limited to the configurations described in the embodiments.

[0014] (First embodiment) An imaging device 100 according to a first embodiment of the present invention will be described below with reference to the accompanying drawings. In this embodiment, when a specific pattern is detected from an invisible light image obtained by imaging, the imaging device 100 switches to a specific control mode among the imaging controls.

[0015] FIG. 1 is a block diagram showing the hardware configuration of an image capturing apparatus 100 according to this embodiment.

[0016] In FIG. 1, an imaging device 100 includes an imaging element 101, a system control unit 102, a pattern recognition unit 103, a recording control unit 104, a strobe control unit 105, a shutter control unit 106, image processing units 110 and 111, and a display unit 114.

[0017] The image sensor 101 can convert an image of a subject into an electrical signal by photoelectric conversion and output it as an image signal. As will be described in detail later with reference to FIG. 2, the image sensor 101 has a plurality of pixels (first pixels) that are sensitive to visible light and a plurality of pixels (second pixels) that are sensitive to invisible light. The image sensor 101 outputs image signals (visible light signals) output from the pixels that are sensitive to visible light to the image processing means 110 as a visible light image 107. The image sensor 101 also outputs image signals (invisible light signals) output from the pixels that are sensitive to invisible light to the image processing means 111 as an invisible light image 108.

[0018] The system control means 102 (control means) controls the entire image capture device 100. A feature of the system control means 102 in this embodiment is that, upon receiving a determination result 109 (a result of pattern recognition) from the pattern recognition means 103, it issues an instruction to switch specific control among the image capture controls of the image capture device 100. Details of the specific control will be described later using FIG. 3, but examples of the specific control include control of the recording control means 104 to record captured still images and videos, control of the strobe light emission operation of the strobe control means 105, and control of the shutter operation of the shutter control means 106. The system control means 102 reads out various programs stored in a storage unit (not shown), and executes arithmetic processing and control processing in accordance with the read out programs.

[0019] When the invisible light image 113 is input from the image processing means 111, the pattern recognition means 103 performs pattern recognition on the invisible light image 113, determines whether or not a specific pattern has been detected, and outputs the determination result 109 to the system control means 102. As the method of pattern recognition is not the subject of the present invention and a detailed description will be omitted, known techniques such as template matching are used.

[0020] The recording control means 104 controls the recording of the visible light image 112 from the image processing means 110 onto a recording medium (not shown). When the specific pattern is detected, the system control means 102 switches the control of the recording control means 104 from control that permits the recording of the visible light image 112 onto the recording medium (first imaging control) to control that prohibits such recording (second imaging control). This makes it possible to control the recording control means 104 so that the visible light image 112 is not recorded onto the recording medium even if the release button (not shown) of the imaging device 100 is pressed in a scene where photography is prohibited. Note that in this embodiment, the recording medium (not shown) onto which the visible light image 112 is recorded is not particularly limited, and examples include a removable memory card, an internal memory (not shown) of the imaging device 100, and an external device (not shown) connected to the imaging device 100 so as to be able to communicate via a communication interface.

[0021] The strobe control means 105 controls the emission of light from a strobe light emitting unit (not shown) for illuminating a subject of the imaging device 100. When the specific pattern is detected, the system control means 102 switches the control of the strobe control means 105 from control that permits emission of light by the strobe light emitting unit (first imaging control) to control that prohibits emission of light by the strobe light emitting unit (second imaging control). This makes it possible to control the imaging device 100 so that flash photography is not possible in scenes where strobe photography is prohibited. The strobe light emitting unit may be built into the imaging device 100, or may be an external device (not shown) that is communicably connected via a communication interface.

[0022] The shutter control means 106 controls a mechanical shutter and an electronic shutter (not shown) used when capturing images with the imaging device 100. When the specific pattern is detected, the system control means 102 switches the control of the shutter control means 106 from control that permits the use of the mechanical shutter (first imaging control) to control that prohibits the use of the mechanical shutter (second imaging control). As a result, when capturing images with the imaging device 100 in a scene that requires silent operation, the electronic shutter is used, and control can be exercised to prevent sound recording.

[0023] When the visible light image 107 is input from the imaging element 101, the image processing means 110 (first image processing means) performs image processing such as gamma correction, white balance processing, and development processing on the input visible light image 107 to generate a visible light image 112. Thereafter, the image processing means 110 outputs the generated visible light image 112 to the recording control means 104 and the display means 114.

[0024] When the non-visible light image 108 is input from the image sensor 101, the image processing means 111 (second image processing means) performs image processing such as gamma correction, white balance processing, and development processing on the input non-visible light image 108 to generate a non-visible light image 113. Thereafter, the image processing means 110 outputs the generated non-visible light image 113 to the pattern recognition means 103. Note that in this embodiment, for the sake of explanation, the image processing means 110 and 111 are divided into two depending on the type of image to be processed, but they may be combined into one. Also, the image sensor 101 separates the image signal into two signals, an image signal for the visible light image 107 and an image signal for the non-visible light image 108, and outputs them to the image processing means 110 and 111, but this is not limiting. For example, a single signal that does not separate the visible light image 107 and the non-visible light image 108 may be output to a single image processing means. In this case, this single image processing means separates the single signal output from the imaging element 101 into image signals for a visible light image 107 and a non-visible light image 108, and performs image processing on each of them to generate a visible light image 112 and a non-visible light image 113.

[0025] The display means 114 is a display device such as a liquid crystal display or an organic EL display that displays the visible light image 112 obtained from the image processing means 110, an operation screen (menu screen) for operating the imaging device 100, etc. Furthermore, as will be described in detail with reference to Fig. 6, when the system control means 102 receives the determination result 109 from the pattern recognition means 103, it displays a mark 602 indicating the determination result 109 on the display means 114 together with the visible light image 112.

[0026] Next, the configuration of the image sensor 101 will be described with reference to FIG.

[0027] FIG. 2(a) is a schematic diagram of a 4*4=16-pixel array extracted from the array of the image sensor 101 for ease of explanation. The image sensor 101 has a photoelectric conversion unit in which photoelectric conversion elements (pixels) typified by photodiodes are arranged two-dimensionally, and color filters 201-204 are provided above each photoelectric conversion element to transmit a specific wavelength band of light from a subject. In FIG. 2(a), the color filter 201 labeled "R" is a color filter that transmits red light, and the color filter 202 labeled "G" is a color filter that transmits green light. In FIG. 2(a), the color filter 204 labeled "B" is a color filter that transmits blue light. These color filters 201, 202, and 204 are regularly arranged in a so-called Bayer array. However, in a normal Bayer array, a color filter that transmits green is also placed at a position that pairs with color filter 202, but in this embodiment, a color filter 203 that transmits infrared light (hereinafter referred to as IR), marked with "IR", is placed at this position.

[0028] In this embodiment, the color filter 203 is a color filter that transmits IR light, but is not limited to this as long as it transmits invisible light. For example, the light that the color filter 203 transmits may be light with a wavelength of 380 nm or less (ultraviolet light). Furthermore, the light that the color filter 203 transmits may be only infrared light with a wavelength of 25,000 nm or more (far-infrared light), only light with a wavelength of 750 nm to 2,500 nm (near-infrared light), or only light with a wavelength of 2,500 nm to 25,000 nm (mid-infrared light). Furthermore, the light that the color filter 203 transmits may be light of multiple invisible wavelengths, for example, infrared light, including near-infrared light and mid-infrared light.

[0029] The light beams transmitted through the color filters 201-204 are received by the pixels arranged directly below the color filters 201-204, undergo photoelectric conversion, and are output from the image sensor 101 as red, blue, green, and IR image signals, respectively. Of these image signals, outputs from the pixels directly below the visible light color filters 201, 202, and 204 are output to the image processing means 110 as the visible light image 107. On the other hand, of these image signals, outputs from the pixels directly below the invisible light color filter 203 are output to the image processing means 111 as the invisible light image 108. The image processing means 110 may perform an interpolation process on the visible light image 107, interpolating the outputs from the pixels where the invisible light color filter 203 is arranged. With this configuration, it is possible to simultaneously obtain the visible light image 107 and the invisible light image 108 from a single image sensor 101.

[0030] In the present embodiment, the color filters 201, 202, and 204 for visible light and the color filter 203 for invisible light are arranged on one image sensor 101, but this is not limiting. For example, the color filters of the image sensor 101 may be arranged in a normal Bayer array, and the color filter for invisible light may be arranged so as to be insertable and removable between the image sensor 101 and the subject. Alternatively, the color filters of the image sensor 101 may be arranged in a normal Bayer array, and the invisible light image 108 may be captured by outputting it from another image sensor having an image sensor arranged only with color filters that transmit only invisible light.

[0031] Next, the photographing process in this embodiment will be described with reference to the flowchart in Fig. 3. This photographing process is characterized by step S303, which switches specific control during image capture, and starts when a power button (not shown) in the image capture device 100 is turned on and so-called live view photographing begins. This photographing process is realized by the system control means 102 executing a control program to control each unit of the image capture device 100.

[0032] In step S301, when the non-visible light image 113 is input from the image processing means 111, the pattern recognition means 103 performs pattern recognition on the non-visible light image 113. Specific patterns to be recognized will be described later.

[0033] In step S302, the pattern recognition means 103 determines whether or not a specific pattern has been detected from the non-visible light image 113 based on the pattern recognition result in step S301, and outputs the determination result 109 to the system control means 102. If a specific pattern has been detected (YES in step S302), the pattern recognition means 103 passes the determination result 109 including the detected specific pattern to the system control means 102, and then proceeds to step S303; if not (NO in step S302), the pattern recognition means 103 proceeds to step S304.

[0034] In step S303, the system control means 102 analyzes the specific pattern detected in step S302, issues a command to switch specific control in accordance with the analysis result, and then proceeds to step S304.

[0035] Here, three examples of specific control according to this embodiment will be described.

[0036] The first specific control is strobe light emission control. Specifically, if a specific pattern is not detected in step S302, the system control means 102 permits the strobe control means 105 to emit strobe light. However, if a specific pattern is detected in step S302, the system control means 102 prohibits the strobe control means 105 from emitting strobe light. This prevents the photographer from accidentally taking pictures with a flash in a scene where flash photography is prohibited.

[0037] The second specific control is shutter control. Specifically, if a specific pattern is not detected in step S302, the system control means 102 permits the shutter control means 106 to transition to mechanical shutter shooting mode. However, if a specific pattern is detected in step S302, the system control means 102 prohibits the shutter control means 106 from transitioning to mechanical shutter shooting mode and permits only transition to electronic shutter shooting mode. This prevents the photographer from accidentally shooting with sound in a shooting scene where sound shooting is prohibited. This is because, compared to a mechanical shutter, an electronic shutter generally allows quieter shooting because it does not control a physical shutter.

[0038] The third specific control is recording control. Specifically, if a specific pattern is not detected in step S302, the system control means 102 permits the recording control means 104 to perform a recording operation to record the shooting results (visible light image 112). However, if a specific pattern is detected in step S302, the system control means 102 prohibits the recording control means 104 from performing this recording operation. This makes it possible to prevent the shooting results from being recorded when the photographer accidentally takes a photo (for example, a photo (or video) that violates privacy).

[0039] Furthermore, the specific control of the above three points involves switching an operation that is permitted before a specific pattern is detected in step S302 (for example, flash emission) to be prohibited after the specific pattern is detected in step S302, but is not limited to this. For example, an operation that is prohibited when a specific pattern is not detected in step S302 (for example, automatic shooting of a visible light image 112 (recording of a live view image)) may be permitted when a specific pattern is detected in step S302.

[0040] In step S304, the system control unit 102 determines whether or not live view shooting is complete after switching to a specific control in step S303, or after determining that a specific pattern has not been detected in step S302. If it is complete (YES in step S304), the system control unit 102 ends the actual shooting process, whereas if it is not complete (NO in step S304), the system control unit 102 returns to step S301.

[0041] Next, with reference to FIG. 4, a description will be given of a specific pattern that is determined to be detected from the non-visible light image 113 in step S302 of FIG.

[0042] Three specific patterns are shown as examples in Figure 4. Note that all of the specific patterns described below are drawn with ink that reflects or emits invisible light (hereinafter referred to as invisible light ink).

[0043] Fig. 4(a) is a diagram showing an example of a specific pattern made up of a mark 401 that prohibits flash emission. When the mark 401 shown in Fig. 4(a) has a specific pattern, by developing (visualizing) the invisible light image 113, the photographer can easily visually verify that the image capture device 100 has switched to the flash emission prohibition mode.

[0044] 4(b) is a diagram showing an example of a specific pattern consisting of a barcode (one-dimensional barcode) 402 that prohibits strobe lighting. By using a barcode as the specific pattern to be placed on the subject in this way, it becomes easy to standardize the size of the pattern and to generate a large number of patterns easily.

[0045] FIG. 4(c) is a diagram showing an example of a specific pattern consisting of a QR code (registered trademark) (two-dimensional barcode) 403 that prohibits strobe lighting. As in the case of FIG. 4(b) where the specific pattern is a barcode, by using a QR code as the specific pattern, it is easy to standardize the pattern size, etc., and it becomes possible to generate a large number of patterns easily. Also, a pattern that combines a barcode and a QR code may be used as the specific pattern.

[0046] Next, with reference to FIG. 5, a method for placing a specific pattern on a subject in this embodiment will be described. In this embodiment, a case where the QR code 403 in FIG. 4(c) is placed on the subject's clothing as the specific pattern will be exemplified. Here, the QR code 403 in FIG. 4(c) is printed in invisible ink on a transparent, colorless sticker, and can be easily attached to the subject's clothing or hat or placed next to a work of art. This sticker is not visible to the naked eye and can be placed anywhere without disrupting the scenery. For example, sponsor marks are usually concentrated in conspicuous areas such as the clothing and helmets worn by motorsport athletes. However, a sticker with the QR code 403 can be freely attached without encroaching on the space for the sponsor marks. Note that instead of a sticker, a specific pattern can also be printed in invisible ink on the subject's clothing, hat, or near a work of art. This is also not visible to the naked eye and can be placed anywhere without disrupting the scenery.

[0047] 5(a) shows a visible light image 112 including a subject 501 displayed as a live view image 500 by the display means 114, and FIG. 5(b) shows a non-visible light image 113. FIG.

[0048] As shown in the visible light image 112 in FIG. 5(a), only the subject 501 is displayed without obstructing the scenery.

[0049] On the other hand, as shown in the invisible light image 113 in FIG. 5(b), it can be seen that a sticker with a QR code 403 drawn in invisible ink as a specific pattern is attached to the jacket of the subject 501. The pattern recognition means 103 performs pattern recognition using this invisible light image 113 as an input. Here, the QR code 403 is detected as a specific pattern, and therefore the pattern recognition means 103 passes a determination result including the detected QR code 403 to the system control means 102. The system control means 102 analyzes the QR code 403 passed from the pattern recognition means 103 and automatically switches specific control in accordance with the analysis result. Here, the QR code 403 is a QR code that prohibits strobe emission, and therefore the system control means 102 switches the strobe emission control, which is the specific control. More specifically, the system control means 102 switches the strobe control means 105 from a strobe emission permission mode to a strobe emission prohibition mode.

[0050] In this way, the image capturing apparatus 100 automatically switches between strobe light emission controls, preventing the photographer from accidentally taking a picture using flash.

[0051] Next, a method for displaying a live view image on a display device by the display means 114 when a specific pattern is detected in this embodiment will be described with reference to Fig. 6. The following describes an example in which the pattern recognition means 103 detects a specific pattern that prohibits flash photography from the invisible light image 113, and the display means 114 displays that the flash emission prohibition mode is active.

[0052] 6 shows the visible light image 112 including the subject 501, displayed as a live view image 600 by the display means 114. In the live view image 600, a mark 602 indicating that the flash mode is disabled is visibly superimposed on the visible light image 112 (FIG. 5(a)) including the subject 501. When the pattern recognition means 103 detects the QR code 403 as a specific pattern in the invisible light image 113, the system control means 102 switches the flash control means 105 to the flash mode disabled and displays the mark 602 on the display means 114. In this way, it is possible to notify the photographer that the mode has been switched.

[0053] Furthermore, the timing for displaying the mark 602 is not limited to the point in time when the pattern recognition means 103 detects the QR code 403 as a specific pattern (first timing). For example, the timing for displaying the mark 602 may be the point in time when the QR code 403 is detected as a specific pattern and the photographer attempts to control strobe light emission (second timing). As the QR code 403 is drawn with invisible ink as described above and therefore cannot be recognized by the naked eye, if the mark 602 suddenly appears in the live view image at the first timing, it may distract the photographer. By making the mark 602 appear in the live view image for the first time at the second timing when the photographer attempts to take an erroneous flash photograph, usability is improved.

[0054] It should be noted that notification means other than the display of the mark 602 may be used. For example, the image capture device 100 may notify the photographer that the mode has been switched using sound, light, vibration, or the like.

[0055] Furthermore, if a specific control of the image capture device 100 has been switched in advance by an operation by the photographer, for example, the image capture device 100 may have been switched in advance to a flash off mode by a user operation. In such a case, even if the pattern recognition means 103 detects the QR code 403 as a specific pattern, the mark 602 may not be displayed in the live view image.

[0056] Next, with reference to FIG. 7 , a method for switching ON / OFF of the readout of the non-visible light signal by the image sensor 101 by the photographer through an operation on a menu screen displayed on a display device will be described. As described above, the image sensor 101 can simultaneously read out visible light signals and non-visible light signals. However, it is assumed that the photographer will set the image sensor 101 not to read out the non-visible light signal in order to save power. This is because, with such a setting, processing blocks such as an AD converter (not shown) that converts the non-visible light signal read out from the image sensor 101 into the non-visible light image 108, the image processing unit 111 that processes the non-visible light image 108, and the pattern recognition unit 103 will not operate, thereby saving power. However, in this case, the pattern recognition of the non-visible light image 113 described above cannot be performed, and therefore the specific control will not be switched. Therefore, it is necessary to warn the photographer that the specific control will not be automatically switched in the image sensor 100.

[0057] FIG. 7 shows an example of a menu screen that the display means 114 displays on the display device when the photographer turns off the reading of the invisible light signal by the image sensor 101.

[0058] When the photographer selects the invisible light off mode switch button 701 on the menu screen, the photographer can turn off the reading of the invisible light signal by the image sensor 101. In this case, the display unit 114 changes the display of the invisible light off mode switch button 701 to "IR OFF ". When switching the display, the display unit 114 also displays a notice 702 on the menu screen to alert the user that certain controls are not automatically restricted in the imaging device 100. For example, as shown in FIG. 7, a statement such as "Please note that flash photography, photography with sound, etc. are not automatically restricted" is displayed as the notice 702 on the menu screen.

[0059] The initial setting for reading out the invisible light signal by the image sensor 101 is ON, and the initial display of the invisible light off mode switch button 701 is "IR ONBy selecting the invisible light off mode switch button 701, the photographer can switch ON / OFF of the reading of the invisible light signal by the image sensor 101. Furthermore, when the photographer selects the invisible light off mode switch button 701 on the menu screen of FIG. 7, the display unit 114 changes the display of the invisible light off mode switch button 701 to "IR ON At this time, the display of the notification 702 on the menu screen also ends.

[0060] As described above, in this embodiment, a colorless, transparent sticker with a specific pattern drawn in invisible light ink is affixed to the subject or to a conspicuous location nearby. Thereafter, when the pattern recognition means 103 recognizes a specific pattern from the invisible light image 113 obtained by live view shooting of the subject, the system control means 102 automatically switches specific control of the image capture device 100 in accordance with the recognized specific pattern. This allows the image capture device 100 to perform accurate pattern recognition without reducing customer satisfaction, and to control image capture according to the shooting scene based on the pattern recognition results.

[0061] (Second embodiment) Next, a second embodiment of the present invention will be described with reference to the accompanying drawings. In this embodiment, an example will be described in which a specific pattern is projected using invisible light by a projection device in a large space such as a stage, a movie theater, or a live concert venue.

[0062] FIG. 8 is a diagram showing a movie theater screen 800 as a subject, which is captured by the imaging device 100 in live view, according to this embodiment.

[0063] As shown in FIG. 8, in this embodiment, a QR code 403 is projected as a specific pattern onto a screen 800 by a projection device 801 using invisible light in a movie theater.

[0064] In the first embodiment of the present invention, it is assumed that a colorless, transparent sticker with a QR code 403 printed in invisible ink is attached to a subject or in a conspicuous location nearby. In this case, a large number of stickers must be prepared for each additional object to be used as a subject, and it is considered time-consuming to print the QR code 403 on the sticker with invisible ink and to attach the sticker to the subject. In addition, it may be better to change the size of the QR code 403 printed as a sticker depending on the size of the subject. For example, if the subject is a car or a train, it is desirable to attach a sticker with a larger QR code 403 printed on it than if the subject is a person. Furthermore, if a sticker is attached to the front of the subject, and the subject faces to the side or back relative to the imaging device 100, the imaging device 100 may not be able to accurately detect the QR code 403 printed on the sticker.

[0065] Therefore, in this embodiment, the QR code 403 is projected onto the screen 800 using invisible light by using the projection device 801. In this way, it becomes possible to project the QR code 403 onto the screen 800 at an optimal size and optimal timing.

[0066] This embodiment can also be applied to outdoor sports such as golf. In this case, it is preferable to project the QR code 403 as a specific pattern onto the subject using invisible light using a known projection technology such as projection mapping. This makes it possible to position the QR code 403 in a position that allows the image capture device 100 to reliably recognize the pattern even if the subject is facing sideways or backwards.

[0067] In this way, the present embodiment can improve the detection accuracy of the pattern recognition means 103 compared to the first embodiment.

[0068] Next, the timing at which the projection device 801 projects the QR code 403 as a specific pattern onto the screen 800 will be described with reference to Fig. 9. In this embodiment, the QR code 403 is projected onto the screen 800 using invisible light, so the amount of light emitted when the projection device 801 projects the QR code 403 onto the screen 800 does not necessarily have to be constant over time.

[0069] For example, as shown in FIG. 9(a), the projection device 801 may be configured to flash invisible light illumination at 1 / 60 second intervals, and the pattern recognition means 103 may be configured to recognize a light emission pattern with a constant cycle in this time direction.

[0070] 9(c), the light emission pattern of the projection device 801 may be an emission pattern with an irregular period in the time direction, such as a ΣΔ modulation method. The light emission pattern of FIG. 9(c) is a specific pattern expressed by the density or width of the light emission time, obtained by ΣΔ modulating a sine wave such as that of FIG. 9(b).

[0071] Next, the photographing process in this embodiment will be described with reference to the flowchart in Fig. 10. Description of the same parts as the photographing process (Fig. 3) executed in the first embodiment will be omitted.

[0072] The difference between this embodiment and the first embodiment is that steps S1002 to S1004 are executed instead of steps S302 and S303.

[0073] In step S1002, the pattern recognition means 103 determines whether or not a QR code 403 has been detected as a specific pattern from the invisible light image 113. If a specific pattern has been detected (YES in step S1002), the process proceeds to step S1003, where a second imaging control process is executed, the mode is switched to flash disabled mode, and the process proceeds to step S304. On the other hand, if a specific pattern has not been detected (NO in step S1002), the process proceeds to step S1004, where a first imaging control process is executed, the mode is switched to flash enabled mode, which is the normal mode, and the process proceeds to step S304. Note that although the flash enabled mode is exemplified as the normal mode here, the normal mode is not limited to this as long as it allows the setting of imaging conditions specified by the photographer. Similarly, if a specific pattern has been detected, the mode is switched to flash disabled mode here, but the mode is not limited to this as long as it sets imaging conditions according to a specific pattern.

[0074] In this way, when the scene of the movie being screened is a special scene in which the projection device 801 projects a specific pattern onto the screen 800, the image capture device 100 automatically controls to restrict (prohibit) a specific operation (such as flash emission). On the other hand, when the scene of the movie being screened is a normal scene in which the projection device 801 does not project a specific pattern onto the screen 800 for a certain period of time, the image capture device 100 automatically controls to return to the initial setting that permits the specific operation (such as flash emission). This eliminates the need for the photographer to reset the mode of the image capture device 100 to the original initial setting every time the movie returns from a special scene to a normal scene, thereby improving usability. In addition, although filming movies (such as secret filming) in movie theaters is generally prohibited, there is a risk that users may intentionally film secretly. Therefore, to prevent secret filming, the projection device 801 continues to project a specific pattern of invisible light onto the screen 800 during the movie. When the pattern recognition means 103 detects a specific pattern from the invisible light image 113, the control of the image capture device 100 switches from allowing the image capture device 100 to record the visible light image 112 on a recording medium to prohibiting such recording. This makes it possible to prevent illegal acts such as surreptitious photography on the image capture device 100 side and to prevent intentional illegal acts by users.

[0075] Also, as in the first embodiment, in this embodiment, when a specific pattern is detected in the non-visible light image 113, the operation permitted by the initial settings of the image capture device 100 is restricted (prohibited), but this is not limited to this. For example, when a specific pattern is detected in the non-visible light image 113, the system control unit 102 may analyze the specific pattern and switch the mode of the image capture device 100 to a setting recommended by the management of a stage, live performance, or other venue. At a stage or live performance venue, there is often a large difference in brightness between the stage and the audience seats, and the lighting often changes suddenly, making it difficult to capture images with the optimal light setting. Furthermore, when the stage or live performance venue is outdoors, the optimal light setting for capture varies depending on the time of day, such as daytime or evening. Conventional cameras were able to set the optimal light setting for capture using photometry, but there are cases where the optimal light metering for each scene cannot always be performed, such as when there is an instantaneous change in lighting.

[0076] Therefore, in a modified example of this embodiment, if the pattern recognition means 103 detects a specific pattern (YES in step S1002), in step S1003, the second imaging control process sets imaging conditions according to the detected specific pattern. Since the venue management knows in advance when the lighting will be switched on and the optimal light intensity depending on the time of day, the projection device 801 can project a specific pattern including the recommended imaging conditions using invisible light at the optimal timing. Imaging conditions include, for example, white balance, shutter speed, aperture, ISO sensitivity, recording, metering mode, etc. For example, the recommended setting may be the optimal metering mode depending on the timing of lighting or the time of day (e.g., daytime or evening). Alternatively, the recommended setting may be a white balance that changes depending on the scenario of a stage performance or live performance. This allows the photographer to capture different atmospheres depending on the scenario. Furthermore, when an actor suddenly starts moving on stage or a player begins a golf swing in a golf tournament, the recommended setting may be a combination of multiple settings, such as aperture, shutter speed, and ISO sensitivity. This makes it possible to reduce blurring when shooting actors or athletes.

[0077] In this way, in the modified example of this embodiment, the settings of the imaging device 100 are automatically switched to recommended settings that the photographer would not normally use, allowing the photographer to enjoy a new photography experience with settings that the photographer has never used before.

[0078] Although it is possible for the photographer to switch the imaging device 100 to the recommended settings if the recommended settings are projected on the screen 800 so as to be visible to the naked eye or are announced in the theater, this is not preferable because it reduces the satisfaction of the photographer, who is the customer. For example, if the QR code 403 is projected on the screen 800 so as to be visible to the naked eye during a movie screening, part of the scene will be hidden by the QR code 403 and will not be visible, and the announcement of the recommended settings during the movie screening will become noise.

[0079] Furthermore, frequently changing the settings of the image capturing device 100 each time the process proceeds to steps S1003 and S1004 may result in a decrease in the quality of the captured image. For this reason, a grace period for changing the settings may be provided, such as by proceeding to step S1004 if a state in which a specific pattern is not detected from the non-visible light image 113 continues for a certain period of time (for example, one second).

[0080] As described above, in this embodiment, by projecting a specific pattern with invisible light, it is possible to prevent the photographer from erroneously taking a photograph using the imaging device 100.

[0081] Furthermore, in a modified example of this embodiment, while the pattern recognition means 103 is detecting a specific pattern from the non-visible light image 113, the imaging device 100 is switched to the recommended settings, allowing the photographer to enjoy a new shooting experience with settings that are not normally used.

[0082] During the period in which the specific pattern is detected, a message indicating that the second imaging control process is being executed in the imaging device 100, such as a message such as "Currently switching to recommended settings," may be displayed on the display means 114. The message may also be notified to the photographer by other notification means, such as sound, light, or vibration.

[0083] (Third embodiment) Next, a third embodiment of the present invention will be described with reference to the accompanying drawings. This embodiment is characterized in that it combines a visible light image 112 and a non-visible light image 113, but does not perform this combination if a specific pattern is detected in the non-visible light image 113.

[0084] FIG. 11 is a block diagram showing the hardware configuration of the imaging device 100 according to this embodiment.

[0085] The imaging device 1100 of this embodiment differs from the imaging device 100 of the first embodiment in that the imaging device 1100 includes an image synthesis unit 1101 that generates a synthesized image by synthesizing a visible light image 112 and a non-visible light image 113. Note that the method of synthesizing the visible light image 112 and the non-visible light image 113 by the image synthesis unit 1101 is a known technique and will not be described in detail here. For example, one example is a method of correcting the visible light image 112 in accordance with the level of a non-visible light luminance signal obtained from the non-visible light image 113.

[0086] In an imaging device capable of simultaneously capturing a visible light image and an invisible light image, for example, in nighttime photography, a composite image obtained by combining the visible light image and the invisible light image may be displayed to improve visibility (see, for example, Japanese Patent Application Laid-Open No. 2014-216734). Also, a composite image obtained by combining the visible light image and the invisible light image may be used to emphasize contrast.

[0087] However, in this embodiment, if the non-visible light image 113 contains a specific pattern, the specific pattern will be reflected in the image after synthesis by the image synthesis means 1101. Therefore, in this embodiment, if the pattern recognition means 103 detects a specific pattern from the non-visible light image 113, the system control means 102 controls the image synthesis means 1101 not to synthesize the visible light image 112 and the non-visible light image 113.

[0088] As described above, in this embodiment, when a specific pattern is detected from the non-visible light image 113, the visible light image 112 and the non-visible light image 113 are not combined, thereby preventing the specific pattern from appearing in the combined image.

[0089] (Fourth embodiment) Next, a fourth embodiment of the present invention will be described with reference to the accompanying drawings as appropriate. In this embodiment, as shown in FIG. 12 , a communication system 1200 will be described as an example, which includes imaging devices 1202 and 1203 and a server 1201 capable of linking the imaging devices 1202 and 1203. The server 1201 may be any information processing device capable of linking multiple imaging devices, such as a smartphone. The communication connection method between the server 1201 and the imaging devices 1202 and 1203 is not particularly limited, and may be a wireless connection or a wired connection. The imaging devices linked by the server 1201 may be devices having the same hardware, which will be described later. While the number of imaging devices is two in this embodiment, it may be three or more.

[0090] In the first embodiment, for example, when subject 501 shown in FIG. 5(b) comes close to the user (photographer) of imaging device 100, imaging device 100 automatically transitions to flash photography prohibited mode only if pattern recognition means 103 can detect QR code 403. However, depending on the positional relationship, such as when imaging device 100 is located behind or beside subject 501, there are cases where pattern recognition means 103 cannot detect QR code 403. In this case, there is a problem in that imaging device 100 cannot automatically transition to flash photography prohibited mode even when live view shooting is being performed close to subject 501.

[0091] In this embodiment, multiple image capture devices cooperate via a server 1201, and when a QR code 403 is detected by the pattern recognition means 103 of one of the image capture devices, not only that image capture device but also other cooperating image capture devices automatically transition to a flash photography prohibited mode. This makes it possible to solve the above-mentioned problem that occurs in the first embodiment.

[0092] FIG. 13 is a block diagram showing the hardware configuration of the image capturing device 1202. As shown in FIG.

[0093] The difference between the imaging device 1202 and the imaging device 100 of the first embodiment is that the imaging device 1202 has communication means 1301 for connecting to the server 1201. Note that the imaging device 1203 has the same hardware configuration as the imaging device 1202, so a description thereof will be omitted. Also, the communication means 1301 can perform communication for cooperation not only with the server 1201 but also among a plurality of imaging devices.

[0094] Fig. 14 is a sequence diagram showing the flow of imaging processing according to this embodiment, which is executed in the server 1201 and the imaging devices 1202 and 1203. Note that Fig. 14 illustrates, but is not limited to, a case where a specific pattern is detected in the imaging device 1202. That is, when a specific pattern is detected in one of the imaging devices linked by the server 1201, the detected specific pattern is transmitted to the remaining imaging devices.

[0095] The imaging process according to this embodiment (FIG. 14) executed in the imaging device 1202 differs from the imaging process according to the first embodiment (FIG. 3) executed in the imaging device 100 in two respects. The first difference is that in step S1400, a communication connection with the server 1201 is established before step S301. The second difference is that if the answer is YES in step S302, the process proceeds to step S304 after step S1401 is executed.

[0096] In step S1400, when the system control unit 102 of the imaging device 1202 starts this process in conjunction with the start of live view shooting, it transmits a communication connection establishment request to the server 1201 and establishes a communication connection with the server 1201. Then, the process proceeds to step S301.

[0097] In step S1401, the system control unit 102 of the imaging device 1202 transmits the specific pattern detected in step S302 to the server 1201 using the communication unit 1301. After that, the process proceeds to step S303.

[0098] Next, a description will be given of the processing in the server 1201 in Fig. 14. In the server 1201, a CPU (not shown) reads out various programs stored in a storage unit (not shown) and executes this processing in accordance with the read out programs.

[0099] First, in step S1402, a communication connection is established with an imaging device (imaging devices 1202 and 1203 in the example of FIG. 14) that has requested establishment of a communication connection. This starts cooperation between the multiple imaging devices (imaging devices 1202 and 1203 in the example of FIG. 14) via the server 1201.

[0100] Next, in step S1403, when a specific pattern is received from one of the image capture devices with which a communication connection was established in step S1402 (the image capture device 1202 in the example of FIG. 14), the process proceeds to step S1404.

[0101] In step S1404, the specific pattern received in step S1403 is transmitted to another imaging device (in the example of FIG. 14, the imaging device 1203) other than the transmission source of the specific pattern in step S1403, among the imaging devices with which a communication connection was established in step S1402. Thereafter, this process ends.

[0102] Next, the processing in the image capture device 1203 in Fig. 14 will be described. This processing starts when a power button (not shown) in the image capture device 1203 is turned on and so-called live view shooting begins. This processing is realized by the system control means 102 of the image capture device 1203 executing a control program to control each unit of the image capture device 1203.

[0103] First, in step S1405, a communication connection establishment request is sent to the server 1201, and a communication connection with the server 1201 is established.

[0104] Next, in step S1406, when a specific pattern is received from the server 1201, the process proceeds to step S1407.

[0105] In step S1407, the specific pattern received from the server 1201 in step S1406 is analyzed, and a specific control switching instruction is issued in accordance with the analysis result, after which the process proceeds to step S1408.

[0106] In step S1408, it is determined whether or not live view shooting is complete after specific control is switched in step S1407. If it is complete (YES in step S1408), the actual shooting process ends, but if it is not complete (NO in step S1408), the process returns to step S1406.

[0107] As described above, according to this embodiment, when a plurality of imaging devices cooperate with each other via the server 1201, if the pattern recognition means 103 of one of the cooperating imaging devices (1202) detects a specific pattern, the specific pattern is transmitted to the other cooperating imaging devices (1203). Just as when a specific pattern is detected by the pattern recognition means 103 of the cooperating imaging device itself, when the specific pattern is transmitted from the server 1201, the cooperating imaging device analyzes the specific pattern and issues a specific control switching instruction according to the analysis result.

[0108] As described above, according to this embodiment, multiple image capturing devices that cooperate via the server 1201 can switch specific controls in all of the image capturing devices if one of the image capturing devices can detect a specific pattern on a sticker attached to the clothing of the subject 501.

[0109] Note that the configuration is not limited to that of this embodiment as long as it has a configuration that produces such effects. For example, in this embodiment, the pattern recognition unit 103 is provided in the image capture devices 1202 and 1203. However, the pattern recognition unit 103 may be provided in the server 1201 instead of the image capture devices 1202 and 1203. In this case, all of the cooperating image capture devices (here, the image capture devices 1202 and 1203) transmit the non-visible light images 113 to the server 1201. In the server 1201, when the pattern recognition unit 103 detects a specific pattern in the non-visible light image 113 received from one of the cooperating image capture devices, the CPU generates a specific control switching instruction. Thereafter, the CPU (instruction transmitting means) transmits the generated specific control switching instruction to all of the cooperating image capture devices. All of the cooperating image capture devices automatically switch the specific control in response to the specific control switching instruction received from the server 1201.

[0110] In this embodiment, the instruction to switch specific control may be an instruction to set the timing of capturing an image when the subject comes directly in front of the image capture device. In this case, when the instruction to switch specific control is executed, all of the linked image capture devices capture an image of the subject 501 when they detect, using the visible light image 112 or the invisible light image 113, that they are positioned to face the subject 501 directly.

[0111] Alternatively, the imaging devices (1202, 1203) may directly communicate with each other without using the server 1201 to switch the specific control. In this case, for example, when a specific pattern is detected by one imaging device (1202), the imaging device (1202) transmits the specific pattern to the other imaging device (1203) via a communication means. When the other imaging device (1203) receives the specific pattern via its own communication means, it performs the same processing as when a specific pattern is detected by its own pattern recognition means 103. That is, the other imaging device (1203) analyzes the specific pattern transmitted from the imaging device (1202) and issues a command to switch the specific control based on the analysis result. The imaging devices may be devices having the same hardware as described above. Although the above description uses two devices, the number of devices may be three or more. The imaging devices that receive the specific pattern and their reception range may be set in advance in the imaging devices.

[0112] (Other embodiments) In this embodiment, a program that implements one or more functions may be provided to a computer in a system or device via a network or storage medium, and the program may be read and executed by a system controller of the system or device. The system controller may have one or more processors or circuits, and may include multiple separate system controllers or a network of multiple separate processors or circuits to read and execute the executable instructions.

[0113] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0114] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0115] 100,1100,1202,1203 Imaging device 1201 Server 101 Image sensor 201~204 Color filters 102 System Control Means 103 Pattern Recognition Tools 104 Recording control means 105 Strobe control means 106 Shutter control means 107,112 visible light images 108,113 invisible light images 109 Judgment result 110,111 Image processing means 114 Display means 1101 Image synthesis means

Claims

1. An imaging device comprising an imaging element having a first pixel that receives visible light from a light beam from a subject and outputs a visible light image, and a second pixel that receives non-visible light from the light beam and outputs a non-visible light image; Control means for switching the imaging control of the visible light image; Pattern recognition means for performing pattern recognition on the non-visible light image; Comprising; The control means switches the imaging control from a first imaging control to a second imaging control when the pattern recognition means detects a specific pattern from the non-visible light image.

2. The imaging device according to claim 1, wherein the imaging element outputs the visible light image to first image processing means and outputs the non-visible light image to second image processing means.

3. The imaging element outputs one signal in which the visible light image and the non-visible light image are not separated to image processing means, The imaging device according to claim 1, wherein the image processing means separates the one signal into image signals of the visible light image and the non-visible light image.

4. The imaging device according to claim 1, wherein the non-visible light is at least one or more of near-infrared light, mid-infrared light, far-infrared light, and ultraviolet light.

5. The control means Executes the second imaging control during a period in which the specific pattern is recognized by the pattern recognition means, The imaging device according to claim 1, wherein control is performed to execute the first imaging control during a period in which the specific pattern is not recognized by the pattern recognition means.

6. Further comprising strobe control means for controlling the light emission of a strobe light emission unit for irradiating the subject, The first imaging control is control for permitting the light emission of the strobe light emission unit to the strobe control means, and the second imaging control is control for prohibiting the light emission of the strobe light emission unit to the strobe control means. The imaging device according to claim 1.

7. Further comprising shutter control means for controlling a mechanical shutter and an electronic shutter, The first imaging control is control for permitting the use of the mechanical shutter to the shutter control means, and the second imaging control is control for prohibiting the use of the mechanical shutter to the shutter control means and permitting only the use of the electronic shutter. The imaging device according to claim 1.

8. further comprising recording control means for controlling the recording of the visible light image, The imaging device according to claim 1, wherein the first imaging control is control for permitting the recording of the visible light image with respect to the recording control means, and the second imaging control is control for prohibiting the recording of the visible light image with respect to the recording control means.

9. The imaging device according to claim 1, wherein the first imaging control is control for permitting the setting of imaging conditions specified by a photographer, and the second imaging control is control for setting the imaging conditions according to the specific pattern.

10. The imaging device according to claim 9, wherein the imaging conditions include at least one or more of white balance, shutter speed, aperture, ISO sensitivity, recording (imaging), and photometry mode.

11. The imaging device according to claim 1, further comprising notification means for notifying a photographer that the second imaging control is being performed when the specific pattern is detected from the non-visible light image by pattern recognition by the pattern recognition means.

12. The imaging device according to claim 11, wherein when the switching from the first imaging control to the second imaging control has been previously performed by a photographer operation, the notification means does not perform the notification even if the specific pattern is detected from the non-visible light image by pattern recognition by the pattern recognition means.

13. The imaging device according to claim 1, further comprising notification means for notifying a photographer that the second imaging control is being performed when the second imaging control is being performed and the photographer attempts to perform control prohibited by the second imaging control.

14. The imaging device according to claim 1, further comprising notification means for notifying a photographer that the second imaging control is being performed during a period in which the specific pattern is detected from the non-visible light image by pattern recognition by the pattern recognition means.

15. further comprising display means, The imaging device according to claim 1, wherein when a photographer sets the imaging device not to use the output of the second pixel, the display means displays that the switching to the second imaging control is not performed.

16. The imaging device according to claim 1, wherein the specific pattern is a specific code.

17. The imaging device according to claim 16, wherein the specific code includes at least one or more of a one-dimensional barcode and a two-dimensional barcode.

18. The imaging device according to claim 1, wherein a seal in which the specific pattern is drawn with invisible ink is attached to the subject or beside the subject.

19. The imaging device according to claim 1, wherein the specific pattern is drawn with invisible ink on the subject or beside the subject.

20. The imaging device according to claim 1, wherein the specific pattern is projected with the non-visible light.

21. The imaging device according to claim 20, wherein when the control means fails to detect a specific pattern from the non-visible light image by pattern recognition by the pattern recognition means, the imaging control is returned from the second imaging control to the first imaging control.

22. The imaging device according to claim 20, wherein when a state in which a specific pattern is not detected from the non-visible light image continues for a certain period by pattern recognition by the pattern recognition means, the imaging control is returned from the second imaging control to the first imaging control.

23. The imaging device according to claim 20, wherein the specific pattern is projected with a light emission pattern having a fixed period.

24. The imaging device according to claim 20, wherein the specific pattern is projected with a light emission pattern having an indefinite period.

25. The imaging device further includes an image synthesizing means for generating a synthesized image of the visible light image and the non-visible light image, The imaging device according to claim 1, wherein when the specific pattern is recognized by the pattern recognition means, the image synthesizing means does not generate the synthesized image.

26. The imaging device according to claim 1, wherein when cooperating with another imaging device having the control means, when a specific pattern is detected from the non-visible light image by pattern recognition by the pattern recognition means, the specific pattern is transmitted to the other imaging device.

27. The imaging device according to claim 26, wherein the cooperation with the other imaging device is executed via an information processing device.

28. An information processing device for coordinating a plurality of imaging devices, When receiving an invisible light image generated from invisible light among the light beams from a subject from one of the plurality of imaging devices, pattern recognition means for performing pattern recognition on the received invisible light image; When the pattern recognition means detects a specific pattern from the received invisible light image, instruction transmission means for transmitting a switching instruction for switching the imaging control of each of the plurality of imaging devices from a first imaging control to a second imaging control to the plurality of imaging devices; An information processing apparatus characterized by comprising the same.

29. A control method for an imaging device, wherein the imaging device comprises an imaging element provided with a photoelectric conversion unit having a first pixel that receives visible light among the light beams from a subject and outputs a visible light image and a second pixel that receives invisible light among the light beams and outputs an invisible light image, and the control method has a control step of switching the imaging control of the visible light image, and a pattern recognition step of performing pattern recognition on the invisible light image, and in the pattern recognition step, when a specific pattern is detected from the invisible light image, in the control step, the imaging control is switched from a first imaging control to a second imaging control.

30. A control method for an information processing apparatus for coordinating a plurality of imaging devices, when receiving an invisible light image generated from invisible light among the light beams from a subject from one of the plurality of imaging devices, a pattern recognition step of performing pattern recognition on the received invisible light image, and in the pattern recognition step, when a specific pattern is detected from the received invisible light image, an instruction transmission step of transmitting a switching instruction for switching the imaging control of each of the plurality of imaging devices from a first imaging control to a second imaging control to the plurality of imaging devices, characterized by comprising the same.

31. A computer-executable program for causing a computer to function as each means of the imaging device according to claim 1.

32. A computer-readable storage medium characterized by storing the program according to claim 31.

33. A computer-executable program for causing a computer to function as each means of the information processing apparatus according to claim 28.

34. A computer-readable storage medium characterized by storing the program according to claim 33.

Images