Imaging device and control method, imaging system

Abstract

This technology provides a method for controlling the position or orientation of a light-emitting device according to the results of determining the video conditions based on the evaluation of captured images. [Solution] The shooting system comprises an imaging device 1 and a light-emitting device 3. The light-emitting device 3, which illuminates the subject with illumination light, is capable of mutual communication with the imaging device 1 and can move or change its orientation according to control commands from the imaging device 1. The imaging device 1 calculates an evaluation value of the image situation from the information of the captured image using a trained model that has undergone machine learning, and performs drive control and light emission control of the light-emitting device 3 based on this evaluation value. If the evaluation value is below a threshold, the imaging device 1 moves the light-emitting device 3 in a predetermined direction, and if the evaluation value becomes above the threshold, it stops the movement of the light-emitting device 3.

Description

Technical Field

[0001] The present invention relates to a technique for controlling the position and orientation of a light-emitting device in a photographing system having an imaging device and a light-emitting device.

Background Art

[0002] In a photographing system using a light-emitting device and an imaging device, when the video situation of a subject changes, it is possible to realize optimal lighting conditions by changing the position and orientation of the light-emitting device. Patent Document 1 discloses a photographing system that can surely move the light-emitting device during photographing. The position of the light-emitting device is determined from the photographing conditions of the imaging unit, the light-emitting ability of the light-emitting unit, and the spatial information acquired by the position information detection unit and the movement area determination unit.

[0003] Patent Document 2 discloses an underwater strobe device used for underwater photography. Based on the actual distance from the camera to the subject and the height from the camera lens to the strobe, the deflection angle of the strobe with respect to the subject is calculated, and control is performed to change the optical axis of the strobe according to the deflection angle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the prior art, the position and deflection angle of the light-emitting device are determined by photographing conditions such as the aperture value and ISO sensitivity, and the installation position of the light-emitting device based on the lens position of the imaging device. Therefore, depending on the photographing situation of the subject, there is a possibility that a suitable image intended by the user cannot be obtained. The object of the present invention is to provide a technology for controlling the position or orientation of a light-emitting device according to the determination result of the video condition based on the evaluation of captured images. [Means for solving the problem]

[0006] The imaging device according to an embodiment of the present invention includes an imaging means for imaging a subject, a detection means for detecting the position or orientation of a light-emitting device that irradiates the subject with illumination light, a calculation means for calculating an evaluation value of the video situation from the image acquired by the imaging means using a machine learning-trained model, and a control means for controlling the position or orientation of the light-emitting device. The control means transmits a control command to the light-emitting device for a position or orientation different from the position or orientation detected by the detection means if the evaluation value is less than a threshold, and performs control to stop changing the position or orientation of the light-emitting device if the evaluation value is equal to or greater than the threshold. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a technology for controlling the position or orientation of a light-emitting device according to the determination result of the video condition based on the evaluation of the captured image. [Brief explanation of the drawing]

[0008] [Figure 1] This is an external view of the imaging system in one embodiment. [Figure 2] This is a block diagram showing a part of the imaging device in the first embodiment. [Figure 3] This is a block diagram showing another part of the imaging device in the first embodiment. [Figure 4] This is a block diagram of the light-emitting device in the first embodiment. [Figure 5] Figure 4 is a block diagram illustrating the drive mechanism. [Figure 6] This is a block diagram showing the configuration of the information processing device in the embodiment. [Figure 7] This is an explanatory diagram showing the data processing flow in the embodiment. [Figure 8] This is a conceptual diagram of the trained model in the embodiment. [Figure 9] This diagram illustrates the input data and training data used for learning. [Figure 10] This diagram illustrates an example of a noisy image. [Figure 11] This is a flowchart illustrating the process in the first embodiment. [Figure 12] Figure 11 is a flowchart illustrating the subsequent processes. [Figure 13] This figure illustrates the changes in video evaluation values ​​over time and the threshold values. [Figure 14] This is a block diagram showing a part of the unmanned imaging device in the second embodiment. [Figure 15] This is a block diagram showing another part of the unmanned imaging device in the second embodiment. [Modes for carrying out the invention]

[0009] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The appearance of the imaging system according to the embodiment will be described with reference to Figure 1. Figure 1(A) is an external view according to the first embodiment. In the first embodiment, an imaging system comprising an imaging device 1 without a drive unit and a light-emitting device 3 without a lens barrel is described. Figure 1(B) is an external view according to the second embodiment. In the second embodiment, an imaging system comprising an imaging unit and a light-emitting unit is described.

[0010] [First Embodiment] Referring to Fig. 1(A), the configuration of the imaging system in this embodiment will be described. Fig. 1 is a diagram schematically showing the imaging system. The imaging system is composed of an imaging device 1 and a light emitting device 3. The imaging device 1 and the light emitting device 3 can communicate with each other. For example, the light emitting device 3 receives control commands such as movement, posture change, and light emission from the imaging device 1. The light emitting device 3 also transmits the acquired information to the imaging device 1. Regarding the communication method between the imaging device 1 and the light emitting device 3, there are a wired communication method and a wireless communication method. The communication method used is appropriately selected according to the usage environment of the imaging system without being limited to either one.

[0011] Referring to Figs. 2 and 3, the configuration of the imaging device 1 will be described. Fig. 2 is a block diagram showing a part of the hardware configuration of the imaging device 1 in Fig. 1. Fig. 3 is a block diagram showing another part of the hardware configuration of the imaging device 1 in Fig. 1. The first control unit 235 has a processor and a memory. For example, the processor is a CPU (Central Processing Unit), MPU (Micro Processing Unit), etc. The memory is a DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc. The first control unit 235 executes various processes to control each component of the imaging device 1 or to control the data transfer between each component.

[0012] The lens barrel unit 201 has an optical member constituting the imaging optical system and an imaging unit 206. The zoom unit 202 includes a zoom lens for performing zooming. The zoom drive control unit 203 drives and controls the zoom unit 202 according to the control command of the first control unit 235. The focus unit 204 includes a lens for performing focus adjustment by a focus adjustment operation. The focus drive control unit 205 drives and controls the focus unit 204 according to the control command of the first control unit 235.

[0013] The imaging unit 206 has an imaging element that performs photoelectric conversion on light from a subject imaged through an imaging optical system. In the imaging unit 206, the imaging element receives light incident through a plurality of lens groups, performs photoelectric conversion, and analog / digital conversion (A / D conversion) of a signal based on the charge corresponding to the amount of the light is performed. The imaging unit 206 outputs the digital signal after A / D conversion to the image processing unit 219.

[0014] The device attitude detection unit 209 detects the attitude of the imaging device 1 and outputs a signal indicating the detection result to the first control unit 235. For example, the device attitude detection unit 209 has the following detection means. · A gyroscope 210 that detects the angular velocity in the direction of rotation around the axes related to the three axes of the imaging device 1. · An accelerometer 211 that detects the acceleration in the three-axis directions of the imaging device 1. · A geomagnetic sensor 212 that detects the orientation of the imaging device 1. The device attitude detection unit 209 performs arithmetic processing regarding the attitude of the imaging device 1, the optical axis of the imaging unit 206, etc. based on the detection signals of the respective detection means.

[0015] The imaging device 1 includes a first power supply unit 213, a second power supply unit 215, and a second control unit 214 provided separately from the first control unit 235. The first power supply unit 213 and the second power supply unit 2!5 supply the power necessary for each part constituting the imaging device 1. The second control unit 214 controls the power supply from the first power supply unit 213 to the first control unit 235. There is no limitation on the hardware configuration of the second control unit 214, and known techniques are used.

[0016] The audio processing unit 216 and the audio input unit 217 perform the acquisition of audio signals and audio commands. The audio input unit 217 acquires audio signals from a microphone (not shown) provided on the imaging device 1 in the surrounding environment of the imaging device 1, performs A / D conversion, and transmits the digital audio signal to the audio processing unit 216. The audio processing unit 216 performs processing related to audio, such as optimizing the input digital audio signal. The audio processing unit 216 can detect the direction of sound on a plane where multiple microphones are installed. The audio signals processed by the audio processing unit 216 are transmitted to the memory 228 (described later) by the first control unit 235 and used for subject search, automatic shooting, etc. Furthermore, the audio processing unit 216 can detect specific audio commands. Regarding audio commands, in addition to using pre-registered commands, users can register specific voices with the imaging device 1.

[0017] The subject distance measuring unit 218 measures the distance between the imaging device 1 and the subject and outputs a signal indicating the measurement result to the first control unit 235. Distance measurement is performed using methods such as time-of-flight distance measurement, phase-difference distance measurement using lasers, or ultrasonic distance measurement. The distance measuring technology used shall be appropriately selected according to the operating environment of the imaging system.

[0018] The image processing unit 219 generates digital image data based on the digital signal received by the imaging unit 206. Furthermore, the image processing unit 219 applies image processing such as distortion correction, white balance adjustment, and color interpolation to the digital image data and outputs the digital image data after the processing is applied. The digital image data output from the image processing unit 219 is converted by the image recording unit 220 into data in a recording format such as JPEG (Joint Photographic Experts Group) format. The converted data is then transmitted to the memory 228 and the video output unit 230, which will be described later.

[0019] The image processing unit 219 and the audio processing unit 216 read the image signal and audio signal temporarily stored in the memory 228, encode the image signal and the audio signal, respectively, to generate a compressed image signal and a compressed audio signal. The first control unit 235 transmits the generated compressed image signal and compressed audio signal to the recording and playback unit 223, which will be described later.

[0020] The learning processing unit 221 in Figure 3 performs machine learning-based processing under the control of the first control unit 235. The learning process will be described later. The recording means in the imaging device 1 consists of a recording medium 222 and a recording / playback unit 223. The recording medium 222 is a recording medium built into the imaging device 1 or a removable recording medium. The recording medium 222 is capable of recording various data corresponding to compressed image signals, compressed audio signals, audio signals, etc., generated by the imaging device 1, and a recording medium with a larger capacity than the non-volatile memory 229 described later is generally used. Various recording media (magnetic disks, optical disks, magneto-optical disks, magnetic tapes, non-volatile semiconductor memory, etc.) can be used.

[0021] The recording and playback unit 223 performs recording processing of predetermined data on the recording medium 222. The predetermined data includes, for example, data corresponding to the compressed image signal and compressed audio signal generated by the image processing unit 219 and the audio processing unit 216, or control data related to shooting. If the audio signal is not compressed, the first control unit 235 transmits the audio signal generated by the audio processing unit 216 and the compressed image signal generated by the image processing unit 219 to the recording and playback unit 223 to record them on the recording medium 222.

[0022] The recording and playback unit 223 performs playback processing of data recorded on the recording medium 222. This data includes, for example, data corresponding to compressed image signals, compressed audio signals, and audio signals, as well as various other data and programs. The first control unit 235 transmits the compressed image signals and compressed audio signals corresponding to the data read from the recording medium 222 by the recording and playback unit 223 to the image processing unit 219 and the audio processing unit 216, respectively. The image processing unit 219 and the audio processing unit 216 temporarily store the data corresponding to the received compressed image signals and compressed audio signals in the memory 228, described later. Then, the image processing unit 219 and the audio processing unit 216 transmit the decoded signals according to a predetermined procedure to the video output unit 230 and the audio output unit 231, described later.

[0023] The first communication unit 224 transmits and receives control signals, etc., between the imaging device 1 and the light-emitting device 3. Control signals are signals related to the control of illumination light irradiation to the subject, such as control of movement and attitude changes of the light-emitting device 3, and light emission commands. The second communication unit 225 transmits and receives control signals, etc., between the imaging device 1 and external devices. Control signals are signals related to commands for starting and ending shooting, and signals related to shooting conditions such as zoom drive. The second communication unit 225 also transmits and receives various parameters related to learning processed by the learning processing unit 221 between the imaging device 1 and external devices.

[0024] The first communication unit 224 and the second communication unit 225 can communicate with the light-emitting device 3 and external devices, respectively, under the control of the first control unit 235. The communication method between the first communication unit 224 and the light-emitting device 3 (communication unit 317 shown in Figure 4) is either wired communication or wireless communication. The communication method used shall be appropriately selected according to the operating environment of the imaging system. Examples of wireless communication include communication conforming to the Bluetooth® standard or wireless LAN communication conforming to the IEEE 802.11 standard in environments where radio wave attenuation is small, such as on land. In environments where radio wave attenuation is large, such as underwater, acoustic communication or optical communication may be used.

[0025] If there are multiple light-emitting devices that can communicate with the imaging device 1, the imaging device 1 communicates with each light-emitting device individually. The communication methods and technologies of the first communication unit 224 and the second communication unit 225 may be the same or different. If the same communication technology is used for the first communication unit 224 and the second communication unit 225, they may be integrated into a single communication unit.

[0026] The device's GPU 226 is a processor responsible for processing video signals and is a processing unit capable of parallel processing of input data. GPU is an abbreviation for "Graphical Processing unit". Alternatively, it is also possible to configure the processing unit using an FPGA. FPGA is an abbreviation for "Field-Programmable Gate Array". The device's GPU 226 can perform calculations efficiently by performing parallel processing of a large amount of data. Therefore, when performing calculations multiple times using machine learning models such as deep learning, parallel processing by the device's GPU 226 is effective. In this embodiment, the device's GPU 226 is used in addition to the first control unit 235 for processing performed by the learning processing unit 221. When the processing of the video discrimination unit 233, which will be described later, is executed, the first control unit 235 and the device's GPU 226 cooperate to perform calculations such as evaluation values ​​(video evaluation values) related to the video situation.

[0027] The setting unit 227 sets a threshold (hereinafter referred to as TH_DRV) that acts as a trigger when operations such as moving the light-emitting device 3, changing its posture, or adjusting the amount of light emitted are performed. The value of TH_DRV is determined based on the evaluation values ​​output by the trained model for video situation discrimination for images used in machine learning (for example, images suitable for the user). For example, the maximum value of the bottom 10% of video evaluation values ​​is set as the threshold and referenced by the first control unit 235.

[0028] Memory 228 temporarily stores data corresponding to image signals and audio signals acquired by, for example, the image processing unit 219 and the audio processing unit 216, respectively. Non-volatile memory (Flash ROM) 229 is electrically erasable and restorable memory. Non-volatile memory 229 stores constants, programs, etc., for the operation of the first control unit 235. Memory 228 and non-volatile memory 229 are connected to the first control unit 235, and control is performed for writing and reading data to these memories as needed.

[0029] The first control unit 235 is connected to a video output unit 230 and an audio output unit 231. The video output unit 230 has a video output terminal and performs processing to display an image signal to an external display or the like connected to the terminal. The audio output unit 231 outputs a pre-set audio pattern from a speaker built into the imaging device 1 during shooting, etc. The video output unit 230 and the audio output unit 231 may be combined into a single terminal, for example, an HDMI (registered trademark: High-Definition Multimedia Interface) terminal.

[0030] The LED control unit 232 controls the LEDs (light-emitting diodes) provided in the imaging device 1 according to the control commands of the first control unit 235. For example, the LED control unit 232 controls the on / off switching of the LEDs in a predetermined blinking pattern during shooting, etc.

[0031] The video discrimination unit 233 determines the video status from the image signal of the image processing unit 219. For example, the video discrimination unit 233 inputs the image signal to a calculation processing unit based on a trained model for video status discrimination, and determines the video status from the output result. The calculation processing unit is realized by the execution of a program by the first control unit 235 and the device GPU 226. The video discrimination unit 233 outputs the discrimination result to the first control unit 235.

[0032] Here, we will explain an example of image condition discrimination in underwater photography. The trained model for image condition discrimination is a trained model that has been trained using images unsuitable for the user and images suitable for the user as training data for the input image data, which is the captured image signal. Images unsuitable for the user (hereinafter referred to as unsuitable images) are images in which color cast or marine snow occurs when shooting in an underwater environment. Marine snow is a phenomenon in which light emitted from the light-emitting device 3 is reflected by floating matter present between the subject and the imaging device 1 in the water, causing it to appear as a white, snow-like substance in the captured image. Images suitable for the user (hereinafter referred to as suitable images) are images in which color cast or marine snow does not occur when shooting in an underwater environment, or images with brightness preferred by the user. The processing unit based on the trained model outputs data that indicates an image condition close to the input image signal.

[0033] The position information detection unit 234 detects the position information of the imaging device 1 and the light-emitting device 3. For example, in an environment where radio wave attenuation is small, such as on land, the position information detection unit 234 is composed of a GPS (Global Positioning System) receiver that receives radio waves from artificial satellites. In an environment where radio wave attenuation is large, such as underwater, the position information detection unit 234 is composed of multiple transducers. The transducers transmit sound waves and receive response signals from transponders. By measuring the arrival angles of the response signals received by the multiple transducers, the first control unit 235 can calculate the position of the light-emitting device 3, which has a transponder, based on the detection results of the position information detection unit 234.

[0034] While the first control unit 235 performs broad control of the imaging device 1, the second control unit 214 (Figure 2) performs control specifically for power supply. The second power supply unit 215 turns ON when external power is supplied from a battery, AC adapter, USB (Universal Serial Bus) terminal, etc., and supplies power to the second control unit 214. With power supplied, the second control unit 214 is always in operation and performs a determination process to decide whether or not to start the first control unit 235 based on predetermined information. The predetermined information includes detection information of pressing the power button provided on the imaging device 1, detection information from the device attitude detection unit 209, input information from the audio processing unit 216, etc. The first power supply unit 213 starts up based on the start determination result of the second control unit 214 and supplies power to the first control unit 235. When the first control unit 235 receives power from the first power supply unit 213, it starts operation according to the start factors of the second control unit 214.

[0035] Next, with reference to Figure 4, the configuration of the light-emitting device 3 in Figure 1 will be described. Figure 4 is a block diagram showing the hardware configuration of the light-emitting device 3. The first control unit 318 has a processor and memory and has the same configuration as the first control unit 235. The first control unit 318 performs various processes to control each component in the light-emitting device 3 and to control data transfer between each component.

[0036] The light-emitting unit 300 is composed of a xenon tube, LED, etc. The light-emitting drive unit 301 generates a drive signal for the light-emitting unit 300 based on the PWM (Pulse Width Modulation) control command and light emission pattern of the first control unit 318. The light-emitting drive unit 301 adjusts the light intensity of the light-emitting unit 300 and drives the light emission on and off. The drive mechanism unit 302 moves and changes the orientation of the light-emitting device 3 according to the control command of the first control unit 318. The configuration of the drive mechanism unit 302 will be described later with reference to Figure 5.

[0037] The device attitude detection unit 308 detects the attitude of the light-emitting device 3 and outputs a signal indicating the detection result to the first control unit 318. For example, the device attitude detection unit 308 has the following detection means. An angular velocity meter 309 detects the angular velocity in the axial direction related to the three axes of the light-emitting device 3. • Accelerometer 310 detects the acceleration of the light-emitting device 3 in three axial directions. • Geomagnetic meter 311 detects the direction of the light-emitting device 3. The device attitude detection unit 308 performs calculations related to the attitude of the light-emitting device 3 and the optical axis of the light-emitting unit 300, etc., based on the detection signals from each detection means.

[0038] The light-emitting device 3 includes a first power supply unit 312, a second control unit 313, and a second power supply unit 314. The first control unit 318 performs broad control of the light-emitting device 3, while the second control unit 313 performs control specifically for power supply. The second control unit 313 controls the power supply to the first control unit 318. The second power supply unit 314 turns ON when external power is supplied from a battery, AC adapter, USB terminal, etc., and supplies power to the second control unit 313. The second control unit 313 is always in operation due to the power supply and performs a determination process to decide whether or not to start the first control unit 318 based on predetermined information. The predetermined information includes detection information of pressing the power button provided on the light-emitting device 3, input information from the imaging device 1, etc. The first power supply unit 312 starts up based on the start determination result of the second control unit 313 and supplies power to the first control unit 318. When the first control unit 318 receives power from the first power supply unit 312, it starts operation according to the start factors of the second control unit 313.

[0039] Memory 315 temporarily stores various types of data. Non-volatile memory (Flash ROM) 316 is electrically erasable and restorable memory. Non-volatile memory 229 stores constants, programs, etc., for the operation of the first control unit 318. Memory 315 and non-volatile memory 316 are connected to the first control unit 318, and control is performed to write and read data to these memories as needed.

[0040] The communication unit 317 communicates with the imaging device 1 under the control of the first control unit 318. The communication method between the first communication unit 224 of the imaging device 1 and the communication unit 317 is as described above. The LED control unit 319 controls the LEDs provided in the light-emitting device 3 according to the control commands of the first control unit 318. For example, the LED control unit 319 controls the on / off status of the LEDs in a predetermined pattern according to the remaining battery level of the second power supply unit 314, etc.

[0041] The position information detection unit 320 detects the position information of the light-emitting device 3. For example, in an environment where radio wave attenuation is small, such as on land, the position information detection unit 320 is composed of a GPS receiver that receives radio waves from artificial satellites. In an environment where radio wave attenuation is large, such as underwater, the position information detection unit 320 is composed of a transponder that receives sound wave signals from a transducer that transmits sound waves and returns a response signal.

[0042] Figure 5 is a block diagram showing an example configuration of the drive mechanism 302. While one example shows a configuration with four actuators, there is no limit to the number of actuators. The drive mechanism 302 comprises a device drive unit 303 and actuators 304 to 307.

[0043] The device drive unit 303 drives actuators 304 to 307 based on the detection values ​​acquired by the device attitude detection unit 308 and the relative position of the light-emitting device 3 transmitted from the imaging device 1 by the communication unit 317. The attitude and position of the light-emitting device 3 can be changed by driving one or more of the first actuator 304, the second actuator 305, the third actuator 306, and the fourth actuator 307. For example, in underwater movement control and attitude control, the propellers corresponding to each actuator are driven to rotate. In land-based movement control and attitude control, the wheels corresponding to each actuator are driven to rotate. In aerial movement control and attitude control, the rotors, propellers, etc., corresponding to each actuator are driven to rotate.

[0044] Next, the configuration of the information processing device will be described with reference to Figure 6. Figure 6 is a block diagram showing the configuration of the information processing device 5 in this embodiment. The information processing device 5 has a hardware configuration that enables communication with the imaging device 1 via a network. An example configuration of a data acquisition server (Figure 7:602) is shown as representative of the information processing device 5.

[0045] The system bus 501 transmits and receives data between the various components (502 to 509) within the information processing device 5. The control center of the information processing device 5 is the CPU 502. The CPU 502 can interpret user instructions and instructions from other devices and execute program code.

[0046] ROM 503 is a non-volatile memory that stores various setting information for the information processing device 5. For example, it is composed of EEPROM or Flash ROM and stores the BIOS program, etc. EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory". BIOS is an abbreviation for "Basic Input Output System". RAM 504 is the main memory of the information processing device 5. RAM 504 is used to temporarily hold data for calculations performed by the CPU 502 and when programs are deployed. For example, RAM 504 is composed of DRAM.

[0047] The HDD (Hard Disk Drive) 505 is an auxiliary storage device for the information processing unit 5. A storage device such as an SSD (Solid State Drive) may be used instead of the HDD 505. The HDD 505 stores the operating system (OS) of the information processing unit 5, programs executed by the CPU 502, various data input to the programs, and the execution results of the programs. The GPU 506 is a processor responsible for processing video signals and other calculations in the information processing unit 5.

[0048] The information and communication unit 507 communicates with external devices. The information and communication unit 507 is composed of, for example, a wireless LAN communication module compliant with the IEEE 802.11 standard, a wired LAN communication module compliant with the IEEE 802.3 standard, etc. The information and communication unit 507 can communicate with the second communication unit 225 of the imaging device 1 via the network.

[0049] The input unit 508 receives user operation instructions for the information processing device 5 and notifies the CPU 502 of the input information. Operation input devices such as keyboards, pointing devices, and touch panels can be used.

[0050] The information display unit 509 has a display device such as an LCD (Liquid Crystal Display). The information display unit 509 can display images according to video signals processed by the GPU 506. The information display unit 509 is configured integrally with the information processing device 5, or it is configured separately from the information processing device 5.

[0051] Referring to Figure 7, the data processing flow related to the learning process will be explained. Figure 7 shows an example of a system configuration comprising a client terminal 601, a data collection server 602, and a learning server 603. Since the client terminal 601 and the learning server 603 have the same hardware configuration as the data collection server 602, a single information processing device may perform multiple functions depending on the scale of the system. In this embodiment, the client terminal 601, the data collection server 602, and the learning server 603 are treated as independent devices. These devices can communicate with each other via a network using their respective information communication units 507 (Figure 6). For example, the client terminal 601, the data collection server 602, the learning server 603, and the imaging device 1 are connected to the internet, enabling mutual communication of data and request signals.

[0052] Before performing an image capture using the imaging system, the user operates the client terminal 601 to select a training image. The training image is an image used by the learning server 603 during training. For example, the training image may be a first image prepared by the user, or a second image presented by the client terminal 601. The second image is an image of a scene captured by the user using the imaging device 1. The client terminal 601 can obtain the data of the second image from the data collection server 602 where the image data is stored, or from an information processing device connected to the internet. For example, the training images may be the aforementioned unsuitable and suitable images. The client terminal 601 transmits the data of the selected training image to the data collection server 602.

[0053] The data acquisition server 602 sends the acquired training image data to the training server 603. The training server 603 uses the acquired training images to calculate video evaluation values ​​and executes the process of generating a trained model for video situation discrimination. The training server 603 sends the data related to the generated trained model (hereinafter referred to as model data) to the data acquisition server 602.

[0054] The imaging device 1 requests the data acquisition server 602 to transmit model data. In response to this transmission request, the data acquisition server 602 transmits the model data to the imaging device 1. The imaging device 1 stores the model data obtained from the data acquisition server 602 in the non-volatile memory 229. The first control unit 235 (and the learning processing unit 221) can evaluate the video being captured by the imaging device 1 by applying the model data. By using the trained model for video condition discrimination, it is possible to obtain an evaluation value that serves as a guideline for whether the video condition is suitable for the user. The position, orientation, or emission of light from the light-emitting device 3 is controlled according to the video evaluation value output based on the information of the captured video and the trained model for video condition discrimination.

[0055] Next, the learning process will be explained with reference to Figures 8 and 9. Figure 8 is a conceptual diagram of the trained model that is machine-learned for each type of subject in this embodiment. In the input / output structure using the trained model, the input node related to the input data 701 in Figure 8 is denoted as "X", and the output node related to the output data 702 in Figure 8 is denoted as "Y". The trained model 703 is a model composed of a neural network.

[0056] The trained model 703 generates and outputs output data 702 in response to input data 701. The trained model for video situation discrimination consists of a network of multilayer perceptrons, which are formed by connecting many neuron models called units.

[0057] The trained model 703 is used to predict output data 702 from input data 701. Prior to this, a training process is performed on the trained video situation discrimination model to output the values ​​of the training output data for the input data. This allows the system to perform a process of inferring output data that follows the trained training data for new input data 701.

[0058] Figure 9 illustrates a table containing input and training data used in machine learning. It shows examples of training data IDs (identification information), input data, and training data. Input data includes shooting scenes (e.g., underwater photography), histogram data for each color (R, G, B) in the image, and noise distribution data. Training data contains information related to the determination of the video situation.

[0059] The data acquisition server 602 (Figure 7) can acquire candidate images to be used as training images from images stored on the data acquisition server 602 or from images on the internet, depending on the shooting scene input by the user to the client terminal 601. Image information such as color histograms and noise distributions are selected as training images.

[0060] The noise distribution will be explained with reference to Figure 10. Figure 10 is an example of a noisy image. For example, consider a scene in underwater photography where floating objects such as marine snow exist between the subject and the imaging device. In this case, the noise distribution is related to noise components such as marine snow. A Gaussian filter is applied to the training image, and the difference between the image before and after filtering is taken to enhance features from the difference image. Noise distribution data is calculated by performing thresholding on the difference image and binarizing the difference image.

[0061] Figure 10 shows the distribution of noise within a rectangular frame in the two-dimensional XY coordinate system set for the captured image. When the coordinate points are represented as (X coordinate, Y coordinate), the four vertices of the rectangular frame are represented as follows. • First vertex (upper left corner): (X coordinate A, Y coordinate A). • Second vertex (upper right corner): (X coordinate B, Y coordinate A). • Third vertex (bottom left corner): (X coordinate A, Y coordinate B). • Fourth vertex (bottom right corner): (X coordinate B, Y coordinate B).

[0062] In Figure 9, the noise distribution column shows the coordinate values ​​in the order of X-coordinate A, X-coordinate B, Y-coordinate A, and Y-coordinate B. The color histogram column shows the luminance value and number of pixels for each color. The color histogram and noise distribution information are used as input data for the learning model. Data representing the video conditions (marine snow, suitable conditions, color cast, etc.) are used as training data. However, various processing methods can be adopted, not limited to the above processing examples for training images.

[0063] The learning server 603 (Figure 7) links the color histogram and noise distribution analyzed for each training image with the video conditions related to the training data. After preparing the input data and training data, the learning server 603 performs machine learning using this data to generate a trained model 703 using a neural network. By using the trained model 703 constructed by machine learning, it is possible to calculate an evaluation value that represents the degree to which the images acquired by the imaging device 1 are images of video conditions suitable for the user.

[0064] In this embodiment, machine learning algorithms are used to generate the trained model 703. Specifically, these include algorithms such as the nearest neighbor method, naive Bayes, decision trees, and support vector machines. There is also deep learning, which uses neural networks to generate features and connection weights for training. Various machine learning algorithms can be applied.

[0065] The processing in the imaging system will be explained with reference to Figures 11 and 12. Figure 11 is a flowchart of the first half of the processing related to the operation of the imaging system. Figure 12 is a flowchart of the second half of the same processing.

[0066] In step S801 of Figure 11, the first control unit 235 of the imaging device 1 performs a determination process to determine whether or not a shooting start signal has been input to the imaging device 1. If it is determined that a shooting start signal has been input, the process proceeds to S802. If it is determined that no shooting start signal has been input, the imaging device 1 enters a shooting standby state, and the determination process in S801 is repeatedly executed.

[0067] In S802, the image processing unit 219 in the imaging device 1 performs image processing on the video data acquired by the imaging unit 206 to generate image data for determining the video status. In accordance with the control commands of the first control unit 235, the device GPU 226 performs calculation processing such as noise distribution and color histogram on the image data generated by the image processing unit 219. Next, the process proceeds to S803.

[0068] In S803, the first control unit 235 and the device GPU 226 calculate video evaluation values ​​linked to the video conditions based on the noise distribution and color histogram calculated in S802 and the trained model 703 for video condition discrimination. For example, the video evaluation values ​​are calculated from the noise distribution range of the video acquired by the imaging unit 206, the color histogram, and the brightness and contrast based on the color histogram. Next, the process proceeds to S804.

[0069] In S804, the video discrimination unit 233 compares the video evaluation value calculated in S803 with its threshold and determines whether the video evaluation value is less than TH_DRV. Figure 13 shows an example of the change in the video evaluation value over time. The horizontal axis represents the passage of time, and the unit time Δt indicates the time interval. The vertical axis represents the video evaluation value, and the horizontal line shows the threshold value, TH_DRV. A higher video evaluation value indicates a video situation in which a more suitable image can be acquired for the user. In the example in Figure 13, the change is shown from a state where the video evaluation value is less than the threshold to a state where the video evaluation value is greater than or equal to the threshold. If it is determined in S804 that the calculated video evaluation value is less than the threshold, the process proceeds to S805. If it is determined that the video evaluation value is greater than or equal to the threshold, the process proceeds to S820 in Figure 12.

[0070] In S805, the imaging device 1 determines the position and orientation of the light-emitting device 3 to continue illuminating the subject. The first control unit 235 calculates the field of view from the distance information to the subject acquired by the subject distance measuring unit 218, the orientation detection information acquired by the device orientation detection unit 209, the size of the imaging unit 206, and the focal length from the imaging unit 206 to the lens. Based on the calculated field of view, the control information for the position and orientation of the light-emitting device 3 is determined.

[0071] For example, it is desirable for the light-emitting device 3 to illuminate the subject from a 45-degree angle above. The position and orientation of the light-emitting device 3 are determined so as to satisfy the following conditions. • The light-emitting device 3 must not be in the field of view of the imaging device 1. The light-emitting device 3 can continue to move while illuminating the subject, with a stopping position that is deviated at a 45-degree angle upward with respect to the optical axis of the imaging device 1.

[0072] An example has been described in which illumination light is shone onto the subject from a 45-degree angle above, but the deviation angle of the optical axis of the imaging device 1 can be changed as appropriate. For example, illumination light may be shone onto the subject from a 45-degree angle in the panning direction, or from a 45-degree angle above to the right or 45-degree angle above to the left. In S805, the first control unit 235 transmits a control command for the position and orientation of the light-emitting device 3 via the first communication unit 224. For example, in the position control command for the light-emitting device 3, its direction of movement is determined according to one or more of the following pieces of information. • Attitude information of the imaging device 1 based on the detection signal from the angular velocity meter 210. • Geomagnetic information based on detection signals from geomagnetic meter 212. • Distance information of the subject measured by the subject distance measuring unit 218. • Field of view of imaging device 1.

[0073] Next, in S806, the light-emitting device 3 moves and changes its orientation according to the position and orientation control commands determined in S805. The first control unit 318 of the light-emitting device 3 controls the device drive unit 303 based on the control commands received by the communication unit 317. The device drive unit 303 drives one or more of the first to fourth actuators 304 to 307, changing the position and orientation of the light-emitting device 3. The process then proceeds to S807.

[0074] In S807, the first control unit 235 determines whether the position of the light-emitting device 3 has reached the stopping position described in S805. Based on the detection information obtained from the position information detection unit 234, the first control unit 235 can detect the position of the light-emitting device 3 relative to the imaging device 1. If it is determined that the light-emitting device 3 has not reached the stopping position, the process proceeds to S808. If it is determined that the light-emitting device 3 has reached the stopping position, the process proceeds to S809.

[0075] In S808, the first control unit 235 compares the current video evaluation value calculated in S803 with previously acquired video evaluation values ​​to determine whether the video evaluation value has decreased while the light-emitting device 3 is moving. If it is determined that the video evaluation value has decreased while the light-emitting device 3 is moving, the process proceeds to S809. If it is determined that the video evaluation value has not decreased while the light-emitting device 3 is moving, the process proceeds to S804.

[0076] In S809, the first control unit 235 performs a process to acquire information on the position and orientation of the light-emitting device 3, which yielded the highest image evaluation value along the movement path of the light-emitting device 3. The first control unit 235 manages historical information related to the movement control and orientation control of the light-emitting device 3, linking it to the image evaluation value. The first control unit 235 can analyze the historical information and image evaluation value to determine the optimal position and orientation of the light-emitting device 3. In the following S810, the first control unit 235 performs a process to transmit control commands for the position and orientation of the light-emitting device 3 via the first communication unit 224, according to the position and orientation acquired in S809. The first control unit 318 of the light-emitting device 3 performs drive control of the light-emitting device 3 based on the control commands received by the communication unit 317. The light-emitting device 3 changes at least one of its position and orientation according to the control commands from the imaging device 1, and stops changing its position or orientation when it reaches the stopping position.

[0077] Next, the process proceeds to S811 in Figure 12, where the imaging device 1 controls the amount of light emitted by the light-emitting device 3. The first control unit 235 transmits a control command to change the amount of light emitted by the light-emitting unit 300 via the first communication unit 224. The first control unit 318 of the light-emitting device 3 controls the light-emitting drive unit 301 according to the control command received from the communication unit 317. The light-emitting drive unit 301 drives the light-emitting unit 300 based on the control command. The amount of light emitted is controlled by switching control such as PWM. For example, in the process of changing the amount of light emitted by the light-emitting unit 300, let's assume that the brightness of the image is below a threshold when the image information is acquired by the imaging unit 206. In this case, to compensate for the lack of brightness, the imaging device 1 performs PWM control to increase the amount of light emitted by the light-emitting unit 300. Next, the process proceeds to S812.

[0078] The processing details for S812 and S813 are the same as those for S804 and S808 in Figure 11, respectively, so a detailed explanation of them will be omitted. If it is determined in S812 that the video evaluation value is less than TH_DRV, the process proceeds to S813. If it is determined that the video evaluation value is TH_DRV or greater, the process proceeds to S820. If it is determined in S813 that the video evaluation value has decreased, the process proceeds to S814. If it is determined that the video evaluation value has not decreased, the process proceeds to S811.

[0079] In S814, the first control unit 235 determines whether or not the light emission amount of the light-emitting unit 300 of the light-emitting device 3 has been switched. For example, consider the case where the light emission amount was increased in S811. In this case, the light emission amount of the light-emitting unit 300 will be switched to a smaller amount. If it is determined in S814 that the light emission amount of the light-emitting unit 300 has not been switched, the process proceeds to S815. If it is determined that the light emission amount of the light-emitting unit 300 has been switched, the process proceeds to S816.

[0080] In S815, the first control unit 235 controls the switching of the light emission amount of the light emission unit 300 based on the change in light emission amount in S811 to S813. The first control unit 318 of the light emission device 3 controls the driving of the light emission unit 300 by the light emission drive unit 301 according to the control command from the imaging device 1. Next, the process proceeds to S811.

[0081] In S816, the first control unit 235 performs a determination process to determine whether the light-emitting device 3 has illuminated the subject from multiple directions. Based on the information of multiple positions and orientations set for the light-emitting device 3 by the control command, the first control unit 235 determines whether the light-emitting device 3 has illuminated the subject from multiple movable positions. If it is determined that the illumination has not occurred, the process proceeds to S817; if it is determined that the illumination has occurred, the process proceeds to S818.

[0082] In S817, the first control unit 235 performs processing to reset the position and orientation of the light-emitting device 3. The first control unit 235 controls the position or orientation of the light-emitting device 3 so that it illuminates the subject from a direction different from the direction corresponding to the position or orientation of the light-emitting device 3 determined in S805. Next, the process proceeds to S805 shown in Figure 11.

[0083] In S818, the first control unit 235 performs a process to acquire information on the position, orientation, and amount of light emitted by the light emitter 3 when the highest image evaluation value was obtained along the movement path of the light emitter 3 during the execution of the processes from S804 to S817. Next, in S819, the first control unit 235 performs a process to transmit control commands for the position, orientation, and amount of light emitted by the light emitter 3 via the first communication unit 224, according to the position, orientation, and amount of light emitted acquired in S818. Based on the control commands received by the communication unit 317, the first control unit 318 of the light emitter 3 performs illumination with the amount of light emitted acquired in S818 using the light emitter 300, and controls the drive mechanism 302 with the position and orientation acquired in S818. When the light emitter 3 reaches the stopping position, it stops changing its position or orientation. Next, the process proceeds to S820.

[0084] In S820, the first control unit 235 performs a notification process to the user regarding the shooting timing. For example, the first control unit 235 controls the video output unit 230 to notify the user of the shooting timing and performs a display process for the notification information. Alternatively, the first control unit 235 uses the LED control unit 232 to notify the user of the shooting timing by the on / off pattern of the LEDs. There are no limitations on the method of notifying the user of the shooting timing, so various methods can be adopted.

[0085] In this embodiment, a suitable image is analyzed, and based on the determination of the image conditions, the position, orientation, or amount of light emitted by the light-emitting device 3 is automatically changed or adjusted. Therefore, it is possible to capture images with appropriate lighting conditions provided by the light-emitting device 3.

[0086] [Second Embodiment] Next, an unmanned imaging device will be described with reference to Figures 1(B), 14, and 15. This embodiment shows an example of a mobile imaging system having the functions of an imaging device 1 and a light-emitting device 3. For example, it can be applied to a drone with imaging capabilities that can move in the air, underwater, or on land. In this embodiment, for components that have the same functions as those described in the first embodiment, the same reference numerals used previously will be reused, and their detailed descriptions will be omitted.

[0087] Figure 1(B) is an external view of the unmanned imaging device 4. The unmanned imaging device 4 has a lens barrel section 201 and a plurality of light-emitting units 400. The unmanned imaging device 4 can move using the first to fourth actuators 404 to 407 and take photographs of subjects. Alternatively, the imaging device 1 and light-emitting unit 3 of the imaging system shown in Figure 1(A) may be replaced by the unmanned imaging device 4, or either one of them.

[0088] Figure 14 is a block diagram showing a part of the hardware configuration of the unmanned imaging device 4. Figure 15 is a block diagram showing another part of the hardware configuration of the unmanned imaging device 4 in Figure 1. The differences from the first embodiment are as follows. • The unit includes a first control unit 435 that has the functions of the first control unit 235 and the first control unit 318. The drive mechanism 402 for moving or changing the orientation of the unmanned imaging device 4 comprises a device drive unit 303 and first to fourth actuators 404 to 407. In the explanation relating to Figure 5, the first to fourth actuators 304 to 307 can be replaced with the first to fourth actuators 404 to 407, respectively. Note that other devices constituting the mobile imaging system (such as the operating device for the unmanned imaging device 4) are not shown or described.

[0089] According to this embodiment, an unmanned imaging device having an imaging unit and a light-emitting unit enables imaging with appropriate lighting conditions.

[0090] In the above embodiment, a shooting system can be provided that allows the position or orientation of the light-emitting device to be changed according to the determination result of the video condition based on the evaluation of the captured image, so that a suitable image can be obtained during shooting.

[0091] The imaging system shown in the above embodiment is just one example, and the operation sequence and search method of the imaging system can be changed as appropriate. The imaging operation may be performed at any timing desired by the user. Alternatively, the imaging device 1 may automatically determine the imaging timing based on the timing in S820 in Figure 12 or the timing of the stopping of movement of the light-emitting device 3. In this embodiment, an example of an imaging system equipped with one light-emitting device 3 has been described, but the imaging system may have multiple light-emitting devices. Although preferred embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications and changes are possible within the scope of its gist.

[0092] Embodiments of this disclosure include the following configurations and methods: [Configuration 1] An imaging means for capturing images of a subject, A detection means for detecting the position or orientation of a light-emitting device that illuminates the subject with illumination light, A calculation means that calculates an evaluation value of the video situation using a machine learning-trained model from the image acquired by the imaging means, It includes control means for controlling the position or orientation of the light-emitting device, If the evaluation value is less than a threshold, the control means transmits a control command to the light-emitting device for a position or orientation different from the position or orientation detected by the detection means. If the evaluation value is equal to or greater than the threshold, the control means performs control to stop changing the position or orientation of the light-emitting device. An imaging device characterized by the following features. [Configuration 2] If the evaluation value is less than the threshold and the control command is used to move the light-emitting device, the control means transmits a control command to the light-emitting device to change its position or orientation to the position or orientation of the light-emitting device that yielded the highest evaluation value along the movement path of the light-emitting device, provided that the light-emitting device has reached the stopping position. The imaging apparatus according to configuration 1, characterized in that it is a device. [Configuration 3] If the evaluation value is below a threshold and the control command is used to move the light-emitting device, the control means transmits a control command to the light-emitting device to change its position or orientation to the position or orientation of the light-emitting device where the highest evaluation value was obtained along the movement path of the light-emitting device, provided that the light-emitting device has not reached the stopping position and the evaluation value has decreased during the movement of the light-emitting device. The imaging apparatus according to configuration 1, characterized in that it is a device. [Structure 4] The aforementioned stopping position is a position in which the light-emitting device does not enter the field of view of the imaging device, or a position having a predetermined deviation angle with respect to the optical axis of the imaging device. The imaging apparatus according to configuration 2 or 3, characterized by the above. [Composition 5] The control means transmits to the light-emitting device a control command for the direction of movement of the light-emitting device, which is determined by one or more of the following: the attitude information of the imaging device detected by the detection means, the geomagnetic detection information, the distance information of the subject acquired by the distance measuring means, and the field of view of the imaging device. An imaging device according to any one of configurations 1 to 4, characterized by the above. [Composition 6] The control means, when the light-emitting device is stopped and the evaluation value is below a threshold, performs control by transmitting a control command to the light-emitting device to change the amount of light emitted by the light-emitting device. An imaging device according to any one of configurations 1 to 5, characterized by the above. [Composition 7] The control means, after the amount of light emitted by the light-emitting device has been changed, if the evaluation value is below a threshold, performs control by transmitting a control command to the light-emitting device to change the amount of light emitted by the light-emitting device. The imaging apparatus according to configuration 6, characterized in that... [Structure 8] If the light emission amount of the light-emitting device is changed and the evaluation value is below a threshold, the control means performs control to reset the position or orientation of the light-emitting device. The imaging apparatus according to configuration 6, characterized in that... [Composition 9] The control means performs a notification process for the shooting timing if the evaluation value is equal to or greater than the threshold. An imaging device according to any one of configurations 1 to 8. [Configuration 10] It has communication means for acquiring data of the trained model through communication with an external device. An imaging device according to any one of configurations 1 to 9, characterized by the above. [Composition 11] An imaging device described in any one of configurations 1 to 10, The light-emitting device comprises the above-mentioned light-emitting device, The control means performs control to change one or more of the position, orientation, and amount of light emitted by the light-emitting device. A shooting system characterized by the following features. [method] A control method performed in an imaging device equipped with imaging means for imaging a subject, A detection step for detecting the position or orientation of a light-emitting device that illuminates the subject with illumination light, A calculation step of calculating an evaluation value of the video situation using a machine learning-trained model from the image acquired by the imaging means, The process includes a control step for controlling the position or orientation of the light-emitting device, In the control step, if the evaluation value is less than the threshold, a control command for a position or orientation different from the position or orientation detected in the detection step is transmitted to the light-emitting device. If the evaluation value is equal to or greater than the threshold, control is performed to stop the change in the position or orientation of the light-emitting device. A control method characterized by the following: [Explanation of Symbols]

[0093] 1. Imaging device 3. Light-emitting device 4. Unmanned imaging device

Claims

1. An imaging means for capturing images of a subject, A detection means for detecting the position or orientation of a light-emitting device that illuminates the subject with illumination light, A calculation means that calculates an evaluation value of the video situation using a machine learning-trained model from the image acquired by the imaging means, It includes control means for controlling the position or orientation of the light-emitting device, The control means, when the evaluation value is less than a threshold, transmits a control command to the light-emitting device for a position or orientation different from the position or orientation detected by the detection means, and when the evaluation value is equal to or greater than the threshold, performs control to stop changing the position or orientation of the light-emitting device. An imaging device characterized by the following features.

2. If the evaluation value is less than the threshold and the control command is used to move the light-emitting device, the control means transmits a control command to the light-emitting device to change its position or orientation to the position or orientation of the light-emitting device that yielded the highest evaluation value along the movement path of the light-emitting device, provided that the light-emitting device has reached the stopping position. The imaging apparatus according to feature 1.

3. If the evaluation value is below a threshold and the control command is used to move the light-emitting device, the control means transmits a control command to the light-emitting device to change its position or orientation to the position or orientation of the light-emitting device where the highest evaluation value was obtained along the movement path of the light-emitting device, provided that the light-emitting device has not reached the stopping position and the evaluation value has decreased during the movement of the light-emitting device. The imaging apparatus according to feature 1.

4. The aforementioned stopping position is a position in which the light-emitting device does not enter the field of view of the imaging device, or a position having a predetermined deviation angle with respect to the optical axis of the imaging device. The imaging device according to claim 2 or 3.

5. The control means transmits to the light-emitting device a control command for the direction of movement of the light-emitting device, which is determined by one or more of the following: the attitude information of the imaging device detected by the detection means, the geomagnetic detection information, the distance information of the subject acquired by the distance measuring means, and the field of view of the imaging device. The imaging apparatus according to feature 1.

6. The control means, when the light-emitting device is stopped and the evaluation value is below a threshold, performs control by transmitting a control command to the light-emitting device to change the amount of light emitted by the light-emitting device. The imaging apparatus according to feature 1.

7. The control means, after the amount of light emitted by the light-emitting device has been changed, if the evaluation value is below a threshold, performs control by transmitting a control command to the light-emitting device to change the amount of light emitted by the light-emitting device. The imaging device according to feature 6.

8. If the light emission amount of the light-emitting device is changed and the evaluation value is below a threshold, the control means performs control to reset the position or orientation of the light-emitting device. The imaging device according to feature 6.

9. The control means performs a notification process for the shooting timing if the evaluation value is equal to or greater than the threshold. The imaging apparatus according to feature 1.

10. It has communication means for acquiring data of the trained model through communication with an external device. The imaging apparatus according to feature 1.

11. The imaging device according to claim 1, The light-emitting device comprises the above-mentioned light-emitting device, The control means performs control to change one or more of the position, orientation, and amount of light emitted by the light-emitting device. A photographic system characterized by the following features.

12. A control method performed in an imaging device equipped with imaging means for imaging a subject, A detection step for detecting the position or orientation of a light-emitting device that illuminates the subject with illumination light, A calculation step of calculating an evaluation value of the video situation using a machine learning-trained model from the image acquired by the imaging means, The process includes a control step for controlling the position or orientation of the light-emitting device, In the control step, if the evaluation value is less than the threshold, a control command for a position or orientation different from the position or orientation detected in the detection step is transmitted to the light-emitting device. If the evaluation value is equal to or greater than the threshold, control is performed to stop the change in the position or orientation of the light-emitting device. A control method characterized by the following:

Images