Imaging support device, imaging device, imaging support method, and program

Abstract

To provide an imaging support device, an imaging device, an imaging support method, and a program that can contribute to reduction in load.SOLUTION: An imaging support device includes an execution part (CPU) and a storage. The storage stores a first learning-completed model 106 to be used for control related to imaging performed by an imaging device. An execution part performs learning processing, using a first image acquired by imaging by the imaging device and a set value applied to the imaging device when the first image is acquired as teacher data, to generate a second learning-completed model 118 to be used for control, and then performs specification processing based upon a first set value 106A output from the first learning-completed model when a second image is input to the first learning-completed model and a second set value 118A output from the second learning-completed model when the second image is input to the second learning-completed model.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an imaging support device, an imaging device, an imaging support method, and a program. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2013-207471 discloses a digital camera comprising: an imaging means for capturing an image of a subject; an imaging control means for controlling imaging processing by the imaging means using imaging processing setting values; an image processing means for performing image processing on image data captured by the imaging means using the image processing setting values; a storage means for storing a table in which feature amounts of a photographed scene in past imaging processing and / or image processing are associated with the imaging processing setting values ​​and / or image processing setting values; a feature calculation means for calculating feature amounts of the photographed scene in the current photograph; and a setting value acquisition means for acquiring the imaging processing setting values ​​and / or image processing setting values ​​based on the feature amounts of the photographed scene calculated by the feature calculation means and the table stored in the storage means. In addition, the digital camera described in JP 2013-207471 A further includes a learning means for learning the features of the shooting scene registered in the table and the imaging processing setting values ​​and / or image processing setting values ​​using a neural network, and the setting value acquisition means acquires the imaging processing setting values ​​and / or image processing setting values ​​based on the learning results by the learning means.

[0003] Japanese Patent Laid-Open Publication No. 2005-347985 discloses a digital camera including an imaging process execution unit that generates image data by executing an imaging process, and an execution control unit that controls the imaging process in accordance with control setting values ​​for one or more setting items, wherein the execution control unit includes a priority determination unit that determines priorities for at least some of a plurality of candidate setting value sets among a plurality of available setting value sets for a setting item set including at least one setting item based on history information related to the setting value sets used in multiple imaging processes, and a setting condition determination unit that determines control setting value sets for the setting item set using the priorities.The digital camera disclosed in Japanese Patent Laid-Open Publication No. 2005-347985 further includes a memory slot that detachably holds a memory, and the priority determination unit has a removable memory history mode that acquires history information from the memory via the memory slot and determines the priority using the acquired history information.

[0004] Japanese Patent Application Laid-Open Publication No. 2003-255428 discloses a camera capable of taking pictures under a plurality of predetermined photographing conditions at least related to brightness, characterized by comprising: a counting means for counting the number of times photographs are taken when photographs are taken under any of the plurality of photographing conditions for each photographing condition; a calculation means for determining a distribution of the count values ​​based on the count values ​​and determining a main photographing condition range as a tendency of the photographing conditions from the determined distribution; and a control means for determining specific photographing conditions within the main photographing condition range other than the plurality of predetermined photographing conditions and for controlling the camera so that photographs can be taken under the determined specific photographing conditions. Summary of the Invention

[0005] An embodiment of the technique of the present disclosure provides an imaging support device, an imaging device, an imaging support method, and a program that can contribute to reducing the load. [Means for solving the problem]

[0006] A first aspect of the technology of the present disclosure is an imaging support device comprising a processor and a memory connected to or built into the processor, wherein the memory stores a first trained model, the first trained model being a trained model used for control related to imaging performed by an imaging device, and the processor performs a learning process using as training data a first image acquired by imaging using the imaging device and setting values ​​applied to the imaging device when the first image was acquired, to generate a second trained model to be used for control, and performs specific processing based on a first setting value output from the first trained model when a second image is input to the first trained model and a second setting value output from the second trained model when the second image is input to the second trained model.

[0007] A second aspect of the technique of the present disclosure is the imaging support device according to the first aspect, in which a second image is stored in the memory.

[0008] A third aspect of the technology of the present disclosure is an imaging support device according to the first or second aspect, in which the processor generates a second trained model by performing a learning process when the condition that the number of first images has reached a first threshold is satisfied.

[0009] A fourth aspect of the technology disclosed herein is an imaging support device according to the third aspect, in which the teacher data is data including a plurality of images obtained by capturing images using an imaging device during a period from a specific time until a condition is satisfied, and a plurality of setting values ​​related to the plurality of images and applied to the imaging device.

[0010] A fifth aspect of the technology disclosed herein is an imaging support device according to the third or fourth aspect, in which the processor performs specific processing based on the first setting value and the second setting value when a condition is satisfied.

[0011] A sixth aspect of the technology disclosed herein is an imaging support device according to any one of the first to fifth aspects, in which a processor performs a specific process when the degree of difference between the first setting value and the second setting value is greater than or equal to a second threshold value.

[0012] A seventh aspect of the technology of the present disclosure is an imaging support device according to any one of the first to sixth aspects, in which a processor performs predetermined processing on the condition that the number of first images reaches a third threshold.

[0013] An eighth aspect of the technology disclosed herein is an imaging support device relating to any one of the first to sixth aspects, in which a processor performs a predetermined processing when the number of first images obtained by imaging in a first environment and used as training data is equal to or greater than a fourth threshold, and the number of first images obtained by imaging in a second environment different from the first environment and used as training data is equal to or less than a fifth threshold.

[0014] A ninth aspect of the technology of the present disclosure is an imaging support device according to any one of the first to eighth aspects, in which the imaging device is an interchangeable lens imaging device and the processor generates multiple second trained models by performing a learning process for each type of interchangeable lens used to capture the first image.

[0015] A tenth aspect of the technology of the present disclosure is an imaging support device according to the ninth aspect, in which, when an interchangeable lens is attached to the imaging device, the processor performs processing using a second trained model, one of multiple second trained models, generated by using an image acquired by capturing an image with the imaging device to which the interchangeable lens is attached as a first image in the learning process.

[0016] An eleventh aspect of the technology of the present disclosure is an imaging support device according to any one of the first to tenth aspects, in which the imaging device includes multiple imaging systems and the processor generates multiple second trained models by performing a learning process for each imaging system used to capture the first image.

[0017] A twelfth aspect of the technology of the present disclosure is an imaging support device according to the eleventh aspect, in which, when an imaging system to be used for imaging is selected from a plurality of imaging systems, a processor performs processing using a second trained model generated by using an image obtained by imaging an imaging device using the selected imaging system as a first image from a plurality of second trained models in the learning process.

[0018] A thirteenth aspect of the technology disclosed herein is an imaging support device according to the twelfth aspect, in which the processor receives an instruction to switch between multiple imaging systems in a stepless manner, and when the instruction is received, continues to use the second trained model assigned to the imaging system before the switch in the imaging system after the switch.

[0019] A fourteenth aspect of the technology of the present disclosure is an imaging support device according to the twelfth or thirteenth aspect, in which a processor uses the scene when the first image was acquired by the imaging device and information related to the selected imaging system as setting values ​​in a learning process, and causes the imaging device to selectively use multiple imaging systems at the startup timing of the imaging device based on a second setting value.

[0020] A 15th aspect of the technology disclosed herein is an imaging support device relating to any one of the first to fourteenth aspects, in which the specific processing is processing including a first processing that reflects a second setting value in the control.

[0021] A 16th aspect of the technology of the present disclosure is an imaging support device according to any one of the first to fifteenth aspects, in which the specific processing includes a second processing of storing the second trained model in a default storage device.

[0022] A 17th aspect of the technology of the present disclosure is an imaging support device according to any one of the first to sixteenth aspects, in which the specific processing includes a third processing that reflects the output of the first trained model or the second trained model, whichever is selected in accordance with instructions received by the processor, in the control.

[0023] An 18th aspect of the technology of the present disclosure is an imaging support device relating to any one of the first to seventeenth aspects, in which the specific processing includes a fourth processing step of outputting first data for displaying on the first display a fourth image corresponding to an image obtained by inputting a third image into the first trained model and applying the first output result output from the first trained model to the third image, and a sixth image corresponding to an image obtained by inputting a fifth image into the second trained model and applying the second output result output from the second trained model to the fifth image.

[0024] A 19th aspect of the technology disclosed herein is an imaging support device related to the 18th aspect, in which the first data includes data for displaying the fourth image and the sixth image on the first display in a distinguishable manner.

[0025] A twentieth aspect of the technology of the present disclosure is an imaging support device according to the eighteenth or nineteenth aspect, in which the first data includes data for displaying on the first display a fourth image in correspondence with first learned model identifying information that can identify the first learned model, and for displaying on the first display a sixth image in correspondence with second learned model identifying information that can identify the second learned model.

[0026] A 21st aspect of the technology of the present disclosure is an imaging support device relating to any one of the 18th to 20th aspects, in which the fourth processing includes processing that reflects the output of the first trained model in the control when the fourth image is selected, and reflects the output of the second trained model in the control when the sixth image is selected, in accordance with instructions received by the processor from the fourth image and sixth image displayed on the first display.

[0027] A 22nd aspect of the technology of the present disclosure is an imaging support device relating to any one of the 1st to 21st aspects, in which the identification process is a process including a 5th process of outputting second data for displaying time identification information on a second display that can identify the time when the second trained model was generated.

[0028] A 23rd aspect of the technology disclosed herein is an imaging support device related to the 22nd aspect, in which the second data includes data for displaying the time-identifying information on the second display in correspondence with a seventh image in which the output of the second trained model is reflected.

[0029] A 24th aspect of the technology of the present disclosure is an imaging support device relating to any one of the 1st to 23rd aspects, in which the identification process is a process including a 6th process of associating the second trained model with time identification information that can identify the time when the second trained model was generated.

[0030] A 25th aspect of the technology of the present disclosure is an imaging support device relating to any one of the first to 24th aspects, in which the specific processing is processing including a seventh processing that reflects the output of the second trained model in the control at a predetermined timing.

[0031] A 26th aspect of the technology of the present disclosure is an imaging support device according to the 25th aspect, in which the predetermined timing is the timing when the imaging device is started, the timing when the number of captured images obtained by imaging by the imaging device becomes equal to or greater than a sixth threshold, the timing when charging of the imaging device begins, the timing when the operating mode of the imaging device transitions from playback mode to setting mode, or the timing when the captured images are rated in playback mode.

[0032] A 27th aspect of the technology of the present disclosure is an imaging support device relating to any one of the 1st to 26th aspects, in which the specific processing is processing including an 8th processing that, when applying the second trained model to another device that is an imaging device different from the imaging device, corrects at least one of the data input to the second trained model and the output from the second trained model based on the characteristics of the imaging device and the characteristics of the other device.

[0033] A 28th aspect of the technology of the present disclosure is an imaging support device according to the 27th aspect, in which the second trained model is accompanied by image sensor information including at least one of characteristic information indicating the characteristics of each of the different image sensors involved in the second trained model and individual difference information indicating individual differences between the different image sensors, and the processor identifies the characteristics of the imaging device and the characteristics of the separate device using the image sensor information.

[0034] A 29th aspect of the technology of the present disclosure is an imaging support device relating to any one of the 1st to 28th aspects, in which the specific processing includes a 9th process of outputting third data for displaying on a third display a first processed image corresponding to an image obtained by inputting an 8th image to a second trained model and applying the third output result output from the second trained model to the 8th image, and an unprocessed image obtained without applying the third output result to the 8th image.

[0035] A 30th aspect of the technology disclosed herein is an imaging support device relating to any one of the first to twenty-ninth aspects, in which the specific processing includes a tenth processing step of inputting a ninth image to a second trained model and outputting fourth data for displaying on a fourth display a brightness-adjusted image in which the fourth output result output from the second trained model is applied to the ninth image to adjust the brightness, and an unprocessed image obtained without applying the fourth output result to the ninth image.

[0036] A 31st aspect of the technology of the present disclosure is an imaging support device relating to any one of the 1st to 30th aspects, in which a third processed image obtained by capturing an image while reflecting the output of the second trained model in the control has first accompanying information attached to the third processed image, and the identification process is a process including an 11th process of including information that can identify the second trained model in the first accompanying information.

[0037] A 32nd aspect of the technology of the present disclosure is an imaging support device relating to any one of the 1st to 31st aspects, in which a fourth processed image obtained by capturing an image while reflecting the output of the first trained model in the control has second accompanying information attached to the fourth processed image, and the identification process is a process including a 12th process of including information that can identify the first trained model in the second accompanying information.

[0038] A 33rd aspect of the technology of the present disclosure is an imaging support device relating to any one of the 1st to 32nd aspects, in which the setting value is at least one of a setting value related to white balance used in imaging, a setting value related to exposure used in imaging, a setting value related to focus used in imaging, a setting value related to saturation used in imaging, and a setting value related to gradation used in imaging.

[0039] A 34th aspect of the technology of the present disclosure is an imaging support device comprising a processor and a memory connected to or built into the processor, wherein the memory stores a first learned model, the first learned model being a learned model used for control related to imaging performed by an imaging device, and the processor performs a learning process using as training data a first image acquired by imaging using the imaging device and setting values ​​applied to the imaging device when the first image was acquired, to generate a second learned model to be used for control, and performs a specific process based on the degree of difference between the first learned model and the second learned model.

[0040] A 35th aspect of the technology of the present disclosure is an imaging device comprising a processor, a memory connected to or built into the processor, and an imaging device main body, wherein the memory stores a first learned model, the first learned model being a learned model used for control related to imaging performed by the imaging device main body, the processor performing a learning process using as training data a first image acquired by imaging by the imaging device main body and setting values ​​applied to the imaging device main body when the first image was acquired, to generate a second learned model to be used for control, and performing a specific process based on a first setting value output from the first learned model when a second image is input to the first learned model and a second setting value output from the second learned model when the second image is input to the second learned model.

[0041] A 36th aspect of the technology of the present disclosure is an imaging assistance method that includes generating a second trained model to be used for control related to imaging performed by the imaging device by performing a learning process using as teacher data a first image acquired by imaging using an imaging device and setting values ​​applied to the imaging device when the first image was acquired, and performing a specific process based on a first setting value output from the first trained model when the second image is input to the first trained model and a second setting value output from the second trained model when the second image is input to the second trained model.

[0042] A 37th aspect of the technology of the present disclosure is a program for causing a computer to execute processing including: generating a second trained model to be used for control related to imaging performed by an imaging device by performing a learning process using as teacher data a first image acquired by imaging using an imaging device and setting values ​​applied to the imaging device when the first image was acquired; and performing a specific process based on a first setting value output from the first trained model when a second image is input to the first trained model and a second setting value output from the second trained model when the second image is input to the second trained model. [Brief explanation of the drawings]

[0043] [Figure 1] 1 is a schematic diagram illustrating an example of the overall configuration of an imaging system. [Figure 2] 1 is a schematic diagram illustrating an example of a hardware configuration of an optical system and an electrical system of an imaging device included in an imaging system. [Figure 3] 2 is a schematic diagram illustrating an example of a hardware configuration of an electrical system of an imaging support device included in the imaging system. FIG. [Figure 4] 2 is a block diagram showing an example of main functions of a CPU included in the imaging support device. FIG. [Figure 5] FIG. 10 is a conceptual diagram illustrating an example of processing content of a teacher data generation unit. [Figure 6] FIG. 10 is a conceptual diagram showing an example of the processing content of the model generation unit when a first trained model is generated. [Figure 7] FIG. 2 is a conceptual diagram illustrating an example of processing contents of a determination unit and a model generation unit. [Figure 8] FIG. 10 is a conceptual diagram showing an example of the processing content of the model generation unit when a second trained model is generated. [Figure 9] FIG. 10 is a conceptual diagram illustrating an example of processing content of an execution unit. [Figure 10] FIG. 10 is a conceptual diagram showing an example of the timing at which verification is performed and the content of the verification. [Figure 11] FIG. 2 is a block diagram showing an example of functions of an execution unit. [Figure 12] FIG. 4 is a conceptual diagram illustrating an example of processing content of a first processing execution unit. [Figure 13] FIG. 10 is a conceptual diagram illustrating an example of processing content of a second processing execution unit. [Figure 14] FIG. 10 is a conceptual diagram illustrating an example of processing content of a third processing execution unit. [Figure 15] FIG. 10 is a conceptual diagram illustrating an example of processing content of a fourth processing execution unit. [Figure 16] FIG. 10 is a screen diagram showing an example of a simulation image display screen displayed on a display under the control of a CPU. [Figure 17]FIG. 10 is a conceptual diagram showing an example of how a simulation image is selected from the simulation image display screen. [Figure 18] This is a conceptual diagram showing an example of processing content when reflecting setting values ​​output from a trained model corresponding to a selected simulation image in control related to imaging. [Figure 19] FIG. 10 is a conceptual diagram illustrating an example of processing content of a fifth processing execution unit. [Figure 20] FIG. 10 is a conceptual diagram illustrating an example of processing content of a sixth processing execution unit. [Figure 21] FIG. 13 is a conceptual diagram showing an example of processing content of a seventh processing execution unit. [Figure 22] FIG. 13 is a conceptual diagram showing an example of processing content of an eighth processing execution unit. [Figure 23] FIG. 13 is a conceptual diagram showing an example of processing content of a ninth processing execution unit. [Figure 24] FIG. 10 is a screen diagram showing an example of a simulation image display screen displayed on a display under the control of a CPU. [Figure 25] FIG. 13 is a conceptual diagram showing an example of processing content of a tenth processing execution unit. [Figure 26] FIG. 10 is a screen diagram showing an example of a simulation image display screen displayed on a display under the control of a CPU. [Figure 27] FIG. 11 is a conceptual diagram showing an example of processing content of an eleventh processing execution unit. [Figure 28] FIG. 13 is a conceptual diagram showing an example of processing content of a twelfth processing execution unit. [Figure 29A] 10 is a flowchart illustrating an example of the flow of an imaging support process. [Figure 29B] This is a continuation of the flowchart shown in FIG. 29A. [Figure 30] FIG. 10 is a conceptual diagram showing an example of the timing at which verification is performed and the content of the verification. [Figure 31] FIG. 2 is a block diagram showing an example of processing content of an execution unit. [Figure 32] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 33]2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 34] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 35] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 36] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 37] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 38] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 39] FIG. 1 is a schematic perspective view showing an example of the external configuration of a smart device. [Figure 40A] FIG. 10 is a conceptual diagram illustrating an example of an aspect in which an instruction to change the angle of view is given to a smart device. [Figure 40B] FIG. 10 is a schematic screen diagram showing an example of a part of a screen used when changing the display magnification. [Figure 41] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 42] 10A and 10B are conceptual diagrams showing an example of a change in the angle of view accompanied by switching of the imaging system midway. [Figure 43] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 44] FIG. 10 is a conceptual diagram illustrating an example of processing content of a model generation unit. [Figure 45] 2 is a block diagram showing an example of processing performed by a CPU of the imaging support device. FIG. [Figure 46] FIG. 2 is a block diagram showing an example of processing performed by a CPU of a smart device. [Figure 47] 10 is a block diagram showing an example of the configuration of an imaging device main body when the imaging device is responsible for the functions of an imaging support device. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0044] Hereinafter, exemplary embodiments of an imaging support device, an imaging device, an imaging support method, and a program according to the techniques of the present disclosure will be described with reference to the accompanying drawings.

[0045] First, the terms used in the following description will be explained.

[0046] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". TPU is an abbreviation for "Tensor processing unit". NVM is an abbreviation for "Non-volatile memory". RAM is an abbreviation for "Random Access Memory". IC is an abbreviation for "Integrated Circuit". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". EL is an abbreviation for "Electro-Luminescence". I / F is an abbreviation for "Interface". UI is an abbreviation for "User Interface". fps is an abbreviation for "frames per second". MF is an abbreviation for "Manual Focus". AF is an abbreviation for "Auto Focus". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". LAN is an abbreviation for "Local Area Network". WAN is an abbreviation for "Wide Area Network". CNN is an abbreviation for "Convolutional Neural Network". AI is an abbreviation for "Artificial Intelligence". Exif is an abbreviation for "exchange image file format".

[0047] As an example, as shown in FIG. 1, an imaging system 10 includes an imaging device 12 and an imaging support device 14. The imaging device 12 is a device that captures an image of a subject. In the example shown in FIG. 1, an interchangeable lens digital camera is shown as an example of the imaging device 12. The imaging device 12 includes an imaging device body 16 and an interchangeable lens 18. The interchangeable lens 18 is interchangeably attached to the imaging device body 16. The interchangeable lens 18 is provided with a focus ring 18A. The focus ring 18A is operated by a user of the imaging device 12 (hereinafter simply referred to as "user") when the user manually adjusts the focus of the imaging device 12 on a subject.

[0048] In this embodiment, an interchangeable lens digital camera is exemplified as the imaging device 12, but this is merely one example, and the imaging device 12 may be a fixed lens digital camera, or a digital camera built into various electronic devices such as a smart device, a wearable terminal, a cell observation device, an ophthalmic observation device, or a surgical microscope.

[0049] The imaging device body 16 is provided with an image sensor 20. The image sensor 20 is a CMOS image sensor. The image sensor 20 captures an image of an imaging range including at least one subject. When an interchangeable lens 18 is attached to the imaging device body 16, subject light representing the subject passes through the interchangeable lens 18 and is focused on the image sensor 20, and image data representing the image of the subject is generated by the image sensor 20.

[0050] In this embodiment, a CMOS image sensor is exemplified as the image sensor 20, but the technology of the present disclosure is not limited to this, and the technology of the present disclosure is also applicable even if the image sensor 20 is another type of image sensor, such as a CCD image sensor.

[0051] A release button 22 and a dial 24 are provided on the top surface of the imaging device body 16. The dial 24 is operated when setting the operation mode of the imaging system and the operation mode of the playback system, and by operating the dial 24, the imaging device 12 is selectively set as an operation mode from among imaging mode, playback mode, and setting mode. The imaging mode is an operation mode that causes the imaging device 12 to capture images. The playback mode is an operation mode that plays back images (e.g., still images and / or moving images) obtained by capturing images for recording in the imaging mode. The setting mode is an operation mode that sets various setting values ​​102 (see FIG. 5), which will be described later, for the imaging device 12.

[0052] The release button 22 functions as an imaging preparation instruction section and an imaging instruction section, and is capable of detecting two stages of pressing operation: an imaging preparation instruction state and an imaging instruction state. The imaging preparation instruction state refers to a state in which the button is pressed from a standby position to an intermediate position (half-pressed position), for example, and the imaging instruction state refers to a state in which the button is pressed beyond the intermediate position to a final pressed position (fully-pressed position). Note that, hereinafter, the "state in which the button is pressed from the standby position to the half-pressed position" will be referred to as the "half-pressed state," and the "state in which the button is pressed from the standby position to the fully-pressed position" will be referred to as the "fully-pressed state." Depending on the configuration of the imaging device 12, the imaging preparation instruction state may be a state in which the user's finger is in contact with the release button 22, and the imaging instruction state may be a state in which the operating user's finger has moved from a state in which the button is in contact with the release button 22 to a state in which the finger is released.

[0053] A touch panel display 32 and instruction keys 26 are provided on the rear surface of the imaging device main body 16.

[0054] The touch panel display 32 includes a display 28 and a touch panel 30 (see also FIG. 2). An example of the display 28 is an EL display (e.g., an organic EL display or an inorganic EL display). The display 28 may be a different type of display, such as a liquid crystal display, instead of an EL display.

[0055] The display 28 displays images and / or text information, etc. When the imaging device 12 is in imaging mode, the display 28 is used to capture images for live view images, i.e., to display live view images obtained by performing continuous imaging. The imaging performed to obtain live view images (hereinafter also referred to as "live view image capture") is performed at a frame rate of, for example, 60 fps. 60 fps is merely an example, and the frame rate may be less than 60 fps or may be greater than 60 fps.

[0056] Here, the term "live view image" refers to a moving image for display based on image data obtained by capturing an image using the image sensor 20. A live view image is also generally referred to as a through image.

[0057] The display 28 is also used to display a still image obtained by capturing an image for a still image when an instruction to capture an image for a still image is given to the imaging device 12 via the release button 22. The display 28 is also used to display a playback image when the imaging device 12 is in a playback mode. Furthermore, the display 28 is also used to display a menu screen on which various menus can be selected when the imaging device 12 is in a setting mode, and a setting screen for setting various setting values ​​102 (see FIG. 5) used in image capture-related controls.

[0058] The touch panel 30 is a transmissive touch panel that is overlaid on the surface of the display area of ​​the display 28. The touch panel 30 receives instructions from the user by detecting contact with a pointing object such as a finger or a stylus pen. For ease of explanation, the "full press state" described above will hereinafter also include a state in which the user presses the soft key for starting imaging via the touch panel 30.

[0059] In this embodiment, an out-cell type touch panel display in which the touch panel 30 is overlaid on the surface of the display area of ​​the display 28 is given as an example of the touch panel display 32, but this is merely an example. For example, an on-cell type or an in-cell type touch panel display can also be used as the touch panel display 32.

[0060] The instruction keys 26 accept various instructions. Here, "various instructions" refers to, for example, an instruction to display a menu screen, an instruction to select one or more menus, an instruction to confirm a selection, an instruction to erase a selection, an instruction to zoom in, zoom out, and frame-by-frame advance. These instructions may also be given via the touch panel 30.

[0061] As will be described in detail later, the imaging device main body 16 is connected to the imaging support device 14 via a network 34. The network 34 is, for example, the Internet. The network 34 is not limited to the Internet and may be a WAN and / or a LAN such as an intranet. In this embodiment, the imaging support device 14 is a server that provides services to the imaging device 12 in response to requests from the imaging device 12. The server may be a mainframe used on-premise with the imaging device 12, or an external server realized by cloud computing. The server may also be an external server realized by network computing such as fog computing, edge computing, or grid computing. Here, a server is given as an example of the imaging support device 14, but this is merely an example, and at least one personal computer or the like may be used as the imaging support device 14 instead of a server.

[0062] As an example, as shown in FIG. 2, the image sensor 20 includes a photoelectric conversion element 72. The photoelectric conversion element 72 has a light-receiving surface 72A. The photoelectric conversion element 72 is disposed within the imaging device body 16 so that the center of the light-receiving surface 72A coincides with the optical axis OA (see also FIG. 1). The photoelectric conversion element 72 has a plurality of photosensitive pixels arranged in a matrix, and the light-receiving surface 72A is formed by the plurality of photosensitive pixels. The photosensitive pixels are physical pixels having photodiodes (not shown), which photoelectrically convert received light and output an electrical signal according to the amount of received light.

[0063] The interchangeable lens 18 includes an imaging lens 40. The imaging lens 40 has an objective lens 40A, a focus lens 40B, a zoom lens 40C, and an aperture 40D. The objective lens 40A, the focus lens 40B, the zoom lens 40C, and the aperture 40D are arranged in this order along the optical axis OA from the subject side (object side) to the imaging device main body 16 side (image side).

[0064] The interchangeable lens 18 also includes a control device 36, a first actuator 37, a second actuator 38, and a third actuator 39. The control device 36 controls the entire interchangeable lens 18 in accordance with instructions from the imaging device body 16. The control device 36 is a device having a computer including, for example, a CPU, NVM, RAM, etc. Note that while a computer is used as an example here, this is merely one example, and devices including ASIC, FPGA, and / or PLD may also be used. The control device 36 may also be a device realized by a combination of hardware and software configurations, for example.

[0065] The first actuator 37 includes a focusing slide mechanism (not shown) and a focusing motor (not shown). The focusing slide mechanism has a focus lens 40B attached thereto so as to be slidable along the optical axis OA. The focusing motor is also connected to the focusing slide mechanism, and the focusing slide mechanism operates by receiving power from the focusing motor to move the focus lens 40B along the optical axis OA.

[0066] The second actuator 38 includes a zoom slide mechanism (not shown) and a zoom motor (not shown). The zoom lens 40C is attached to the zoom slide mechanism so that it can slide along the optical axis OA. The zoom motor is also connected to the zoom slide mechanism, and the zoom slide mechanism operates by receiving power from the zoom motor to move the zoom lens 40C along the optical axis OA.

[0067] The third actuator 39 includes a power transmission mechanism (not shown) and an aperture motor (not shown). The aperture 40D has an opening 40D1, the size of which is variable. The opening 40D1 is formed by a plurality of aperture blades 40D2. The plurality of aperture blades 40D2 are connected to the power transmission mechanism. The power transmission mechanism is also connected to an aperture motor, which transmits the power of the aperture motor to the plurality of aperture blades 40D2. The plurality of aperture blades 40D2 operate in response to the power transmitted from the power transmission mechanism, thereby changing the size of the aperture 40D1. The aperture 40D adjusts the exposure by changing the size of the opening 40D1.

[0068] The focus motor, zoom motor, and aperture motor are connected to a control device 36, which controls the driving of each of the focus motor, zoom motor, and aperture motor. In this embodiment, a stepping motor is used as an example of the focus motor, zoom motor, and aperture motor. Therefore, the focus motor, zoom motor, and aperture motor operate in synchronization with pulse signals in response to commands from the control device 36. While an example is shown here in which the focus motor, zoom motor, and aperture motor are provided in the interchangeable lens 18, this is merely an example, and at least one of the focus motor, zoom motor, and aperture motor may be provided in the imaging device body 16. The components and / or operation method of the interchangeable lens 18 can be changed as needed.

[0069] In the imaging mode, the imaging device 12 selectively sets MF mode and AF mode in accordance with instructions given to the imaging device body 16. MF mode is an operating mode in which the focus is adjusted manually. In MF mode, for example, when the user operates the focus ring 18A or the like, the focus lens 40B moves along the optical axis OA by an amount corresponding to the amount of operation of the focus ring 18A or the like, thereby adjusting the focus.

[0070] In AF mode, the imaging device body 16 calculates the in-focus position according to the subject distance and adjusts the focus by moving the focus lens 40B toward the calculated in-focus position. Here, the in-focus position refers to the position of the focus lens 40B on the optical axis OA when the subject is in focus. Note that, for ease of explanation, the control for adjusting the focus lens 40B to the in-focus position will also be referred to as "AF control" below.

[0071] The imaging device main body 16 includes an image sensor 20, a controller 44, an image memory 46, a UI device 48, an external I / F 50, a communication I / F 52, a photoelectric conversion element driver 54, a mechanical shutter driver 56, a mechanical shutter actuator 58, a mechanical shutter 60, and an input / output interface 70. The image sensor 20 also includes a photoelectric conversion element 72 and a signal processing circuit 74.

[0072] The input / output interface 70 is connected to the controller 44, image memory 46, UI device 48, external I / F 50, photoelectric conversion element driver 54, mechanical shutter driver 56, and signal processing circuit 74. The input / output interface 70 is also connected to the control device 36 of the interchangeable lens 18.

[0073] The controller 44 includes a CPU 62, an NVM 64, and a RAM 66. The CPU 62, the NVM 64, and the RAM 66 are connected via a bus 68, which is connected to an input / output interface 70.

[0074] 2, for convenience of illustration, one bus is shown as bus 68, but multiple buses may be used. Bus 68 may be a serial bus or a parallel bus including a data bus, an address bus, a control bus, etc.

[0075] The NVM 64 is a non-transitory storage medium that stores various parameters and programs. For example, the NVM 64 is an EEPROM. However, this is merely an example, and instead of or together with the EEPROM, an HDD and / or an SSD may be used as the NVM 64. The RAM 66 temporarily stores various information and is used as a work memory.

[0076] The CPU 62 reads out necessary programs from the NVM 64 and executes the read programs in the RAM 66. The CPU 62 controls the entire imaging device 12 in accordance with the programs executed on the RAM 66. In the example shown in Fig. 2, the image memory 46, the UI device 48, the external I / F 50, the communication I / F 52, the photoelectric conversion element driver 54, the mechanical shutter driver 56, and the control device 36 are controlled by the CPU 62.

[0077] The photoelectric conversion element 72 is connected to a photoelectric conversion element driver 54. The photoelectric conversion element driver 54 supplies an imaging timing signal that defines the timing of imaging performed by the photoelectric conversion element 72 to the photoelectric conversion element 72 in accordance with an instruction from the CPU 62. The photoelectric conversion element 72 performs resetting, exposure, and output of an electrical signal in accordance with the imaging timing signal supplied from the photoelectric conversion element driver 54. Examples of imaging timing signals include a vertical synchronization signal and a horizontal synchronization signal.

[0078] When the interchangeable lens 18 is attached to the imaging device body 16, subject light incident on the imaging lens 40 is imaged on the light-receiving surface 72A by the imaging lens 40. Under the control of the photoelectric conversion element driver 54, the photoelectric conversion element 72 photoelectrically converts the subject light received by the light-receiving surface 72A and outputs an electrical signal corresponding to the amount of subject light to the signal processing circuit 74 as analog image data indicating the subject light. Specifically, the signal processing circuit 74 reads out the analog image data from the photoelectric conversion element 72 in units of one frame and for each horizontal line using an exposure sequential readout method.

[0079] The signal processing circuit 74 generates digital image data by digitizing the analog image data. Note that, for the sake of convenience, hereinafter, when there is no need to distinguish between the digital image data that is the subject of internal processing in the imaging device main body 16 and the image represented by the digital image data (i.e., the image visualized based on the digital image data and displayed on the display 28, etc.), they will be referred to as the "captured image 75."

[0080] The mechanical shutter 60 is a focal plane shutter and is disposed between the aperture 40D and the light receiving surface 72A. The mechanical shutter 60 includes a front curtain (not shown) and a rear curtain (not shown). Each of the front curtain and the rear curtain includes multiple blades. The front curtain is disposed closer to the subject than the rear curtain.

[0081] The mechanical shutter actuator 58 is an actuator having a link mechanism (not shown), a solenoid for a first curtain (not shown), and a solenoid for a second curtain (not shown). The solenoid for the first curtain is the drive source for the first curtain and is mechanically linked to the first curtain via the link mechanism. The solenoid for the second curtain is the drive source for the second curtain and is mechanically linked to the second curtain via the link mechanism. The mechanical shutter driver 56 controls the mechanical shutter actuator 58 in accordance with instructions from the CPU 62.

[0082] The first curtain solenoid generates power under the control of the mechanical shutter driver 56 and selectively winds up or lowers the first curtain by applying the generated power to the first curtain. The second curtain solenoid generates power under the control of the mechanical shutter driver 56 and selectively winds up or lowers the second curtain by applying the generated power to the second curtain. In the imaging device 12, the opening and closing of the first curtain and the second curtain are controlled by the CPU 62, thereby controlling the amount of exposure to the photoelectric conversion element 72.

[0083] In the imaging device 12, imaging for live view images and imaging for recording images for recording still images and / or moving images are performed using an exposure sequential readout method (rolling shutter method). The image sensor 20 has an electronic shutter function, and imaging for live view images is achieved by activating the electronic shutter function without operating the mechanical shutter 60, which is left fully open.

[0084] In contrast, imaging involving actual exposure, i.e., imaging for a still image, is achieved by activating the electronic shutter function and operating the mechanical shutter 60 so as to transition the mechanical shutter 60 from a front curtain closed state to a rear curtain closed state.

[0085] The image memory 46 stores the captured image 75 generated by the signal processing circuit 74. That is, the signal processing circuit 74 stores the captured image 75 in the image memory 46. The CPU 62 acquires the captured image 75 from the image memory 46 and executes various processes using the acquired captured image 75.

[0086] The UI device 48 includes a display 28, and the CPU 62 displays various pieces of information on the display 28. The UI device 48 also includes a reception device 76. The reception device 76 includes a touch panel 30 and a hard key unit 78. The hard key unit 78 is a plurality of hard keys including the instruction keys 26 (see FIG. 1 ). The CPU 62 operates in accordance with various instructions received by the touch panel 30. Note that, although the hard key unit 78 is included in the UI device 48 here, the technology of the present disclosure is not limited to this, and for example, the hard key unit 78 may be connected to the external I / F 50.

[0087] The external I / F 50 controls the exchange of various information with devices (hereinafter also referred to as "external devices") that exist outside the imaging device 12. An example of the external I / F 50 is a USB interface. To the USB interface, external devices (not shown) such as smart devices, personal computers, servers, USB memory, memory cards, and / or printers are directly or indirectly connected.

[0088] The communication I / F 52 controls the exchange of information between the CPU 62 and the imaging support device 14 (see FIG. 1) via the network 34 (see FIG. 1). For example, the communication I / F 52 transmits information in response to a request from the CPU 62 to the imaging support device 14 via the network 34. The communication I / F 52 also receives information transmitted from the imaging support device 14 and outputs the received information to the CPU 62 via the input / output interface 70.

[0089] 3, the imaging support device 14 includes a computer 82 and a communication I / F 84. The computer 82 includes a CPU 86, a storage 88, and a memory 90. Here, the computer 82 is an example of a "computer" according to the technology of the present disclosure, the CPU 86 is an example of a "processor" according to the technology of the present disclosure, and the storage 88 is an example of a "memory" according to the technology of the present disclosure.

[0090] The CPU 86, storage 88, memory 90, and communication I / F 84 are connected to a bus 92. Although the example shown in Fig. 3 shows a single bus as the bus 92 for convenience of illustration, multiple buses may be used. The bus 92 may be a serial bus or a parallel bus including a data bus, an address bus, a control bus, etc.

[0091] The CPU 86 controls the entire imaging support device 14. The storage 88 is a non-volatile storage device that stores various programs, various parameters, etc. Examples of the storage 88 include non-transitory storage media such as an EEPROM, an SSD, and / or an HDD. The memory 90 is a memory that temporarily stores information and is used as a work memory by the CPU 86. Examples of the memory 90 include a RAM.

[0092] The communication I / F 84 is connected to the communication I / F 52 of the imaging device 12 via the network 34. The communication I / F 84 controls the exchange of information between the CPU 86 and the imaging device 12. For example, the communication I / F 84 receives information transmitted from the imaging device 12 and outputs the received information to the CPU 86. The communication I / F 84 also transmits information in response to a request from the CPU 86 to the imaging device 12 via the network 34.

[0093] The imaging support device 14 includes a backup storage device 94. The backup storage device 94 is an example of a "default storage device" according to the technology of the present disclosure. The backup storage device 94 is a non-volatile storage device that stores a second trained model 118 (see FIG. 8 ), which will be described later, and the like. Examples of the backup storage device 94 include non-transitory storage media such as an EEPROM, an SSD, and / or an HDD. The backup storage device 94 is connected to the bus 92, and the CPU 86 stores the second trained model 118, which will be described later, in the backup storage device 94 and reads the second trained model 118, which will be described later, from the backup storage device 94.

[0094] Incidentally, one type of conventionally known imaging device is one that is equipped with an automatic setting function that sets various parameters used in image capture-related controls (for example, parameters used for exposure correction, parameters used for AF control, parameters used for gradation correction, etc.) in accordance with various conditions. However, the various parameters used in this type of imaging device are merely parameters determined by the manufacturer according to its own standards, and it is difficult to say that the parameters reflect the preferences of each user.

[0095] Therefore, in this embodiment, the imaging support device 14 performs machine learning on a learning model regarding the relationship between the setting values ​​actually used in imaging using the imaging device 12 and the captured image 75 acquired by imaging by the imaging device 12, and by using the trained model, supports setting parameters for the imaging device 12 that are as close as possible to the user's preferences. Furthermore, the imaging support device 14 performs processing that takes into consideration the timing of performing processing using the learning results. A specific example will be described below.

[0096] 4, the storage 88 of the imaging support device 14 stores an imaging support processing program 96. The imaging support processing program 96 is an example of a "program" according to the technology of the present disclosure.

[0097] The CPU 86 reads the imaging support processing program 96 from the storage 88, and executes the read imaging support processing program 96 on the memory 90. The CPU 86 performs imaging support processing in accordance with the imaging support processing program 96 executed on the memory 90 (see also FIGS. 29A and 29B). The imaging support processing is realized by the CPU 86 operating as a teacher data generation unit 86A, a model generation unit 86B, a determination unit 86C, and an execution unit 86D.

[0098] Hereinafter, an example of specific processing contents by the teacher data generating unit 86A, the model generating unit 86B, the determining unit 86C, and the executing unit 86D will be described with reference to FIGS.

[0099] As an example, as shown in FIG. 5, the teacher data generation unit 86A generates teacher data 98. The teacher data 98 ​​is labeled data used in machine learning. In this embodiment, the teacher data 98 ​​is used in the learning process for the CNN 104 (see FIG. 6), the replication model 116 (see FIG. 8), and the second trained model 118 (see FIG. 8), etc. The teacher data 98 ​​includes the captured image 75 and the correct answer data 100. Note that the "trained model" according to the technology of the present disclosure also includes a model that can be further trained.

[0100] The NVM 64 of the imaging device 12 (see FIG. 2) stores parameters applied to the imaging device 12, i.e., various setting values ​​102 applied to control related to imaging performed by the imaging device. Note that, for convenience of explanation, hereinafter, control related to imaging performed by the imaging device will also be simply referred to as "control related to imaging."

[0101] The various setting values ​​102 include, for example, a setting value for white balance R (e.g., a white balance gain applied to red (R)), a setting value for white balance B (e.g., a white balance gain applied to blue (B)), a setting value used for exposure correction, a setting value used for adjusting high color tones according to the imaging scene, a setting value used for adjusting shadow tones according to the imaging scene, and a setting value used for adjusting colors according to the imaging scene.

[0102] Note that the setting values ​​for white balance R and white balance B are examples of "setting values ​​related to white balance used in imaging" according to the technology of the present disclosure. The setting values ​​used for exposure correction are examples of "setting values ​​related to exposure used in imaging" according to the technology of the present disclosure. The setting values ​​used for adjusting high color tones (also referred to as "highlight tones") according to the imaging scene and the setting values ​​used for adjusting shadow tones according to the imaging scene are examples of "setting values ​​related to gradation used in imaging" according to the technology of the present disclosure. The setting values ​​used for adjusting colors according to the imaging scene are examples of "setting values ​​related to saturation used in imaging" according to the technology of the present disclosure.

[0103] The teacher data generation unit 86A acquires a captured image 75 from the image memory 46 of the imaging device 12 (see FIG. 2). The teacher data generation unit 86A also acquires various setting values ​​102 corresponding to the acquired captured image 75, i.e., various setting values ​​102 applied to control related to imaging performed to obtain the captured image 75, from the NVM 64. The setting values ​​102 applied to the imaging device 12 are values ​​set by the user.

[0104] The teacher data generation unit 86A uses the various setting values ​​102 acquired from the NVM 64, i.e., the various setting values ​​102 applied to the imaging device 12 when the captured image 75 was acquired by the imaging device 12, as supervised answer data 100. The teacher data generation unit 86A generates teacher data 98 ​​by associating the captured image 75 acquired from the image memory 46 with the supervised answer data 100.

[0105] The teacher data generation unit 86A generates the teacher data 98 ​​each time a user takes an image, that is, each time a captured image 75 is stored in the image memory 46. In the following, the number of images may be referred to as the "number of frames" or "number of images."

[0106] The teacher data generation unit 86A stores the generated teacher data 98 ​​on an image-by-image basis in the storage 88. A plurality of teacher data 98 ​​are stored in the storage 88. That is, a plurality of captured images 75 and a plurality of pieces of supervised answer data 100 associated with the plurality of captured images 75 are stored in the storage 88 as a plurality of teacher data 98.

[0107] As an example, as shown in FIG. 6 , the model generation unit 86B includes a CNN 104. The model generation unit 86B acquires training data 98 ​​from the storage 88 and inputs a captured image 75 included in the acquired training data 98 ​​to the CNN 104. When the captured image 75 is input, the CNN 104 outputs a CNN signal 104A corresponding to various setting values ​​102. The CNN signal 104A is a signal indicating setting values ​​of items similar to the various setting values ​​102. Examples of setting values ​​of items similar to the various setting values ​​102 include a setting value for white balance R, a setting value for white balance B, a setting value used for exposure correction, a setting value used for adjusting high color tones according to the imaging scene, a setting value used for adjusting shadow tones according to the imaging scene, and a setting value used for adjusting colors according to the imaging scene.

[0108] The model generation unit 86B calculates an error 108 between the ground truth data 100 associated with the captured image 75 input to the CNN 104 and the CNN signal 104A. The error 108 refers to, for example, an error in the setting value of white balance R, an error in the setting value of white balance B, an error in the setting value used for exposure correction, an error in the setting value used for adjusting high color tones according to the captured scene, an error in the setting value used for adjusting shadow tones according to the captured scene, and an error in the setting value used for adjusting colors according to the captured scene.

[0109] The model generation unit 86B calculates a plurality of adjustment values ​​110 that minimize the error 108. Then, the model generation unit 86B adjusts a plurality of optimization variables in the CNN 104 using the calculated plurality of adjustment values ​​110. Here, the plurality of optimization variables in the CNN 104 refers to, for example, a plurality of connection weights and a plurality of offset values ​​included in the CNN 104.

[0110] The model generation unit 86B repeatedly performs a learning process, including inputting the captured image 75 to the CNN 104, calculating the error 108, calculating multiple adjustment values ​​110, and adjusting multiple optimization variables in the CNN 104, for the number of captured images 75 stored in the storage 88. That is, the model generation unit 86B optimizes the CNN 104 by adjusting multiple optimization variables in the CNN 104 using the multiple adjustment values ​​110 calculated so as to minimize the error 108 for each of the multiple captured images 75 in the storage 88. Note that the model generation unit 86B does not necessarily need to be provided inside the CPU 86 and may be provided outside the CPU 86. That is, the learning process is not limited to being performed by the CPU 86 itself, and the learning process also includes a process of causing the model generation unit 86B, which is provided outside the CPU 86, to perform the learning process under the control of the CPU 86 to generate a trained model.

[0111] The model generation unit 86B generates the first trained model 106 by optimizing the CNN 104. That is, the CNN 104 is optimized by adjusting a plurality of optimization variables included in the CNN 104, thereby generating the first trained model 106. The model generation unit 86B stores the generated first trained model 106 in the storage 88. As will be described in detail later, the first trained model 106 is a trained model used for control related to imaging. Note that this control related to imaging includes not only control related to image capture by the image sensor 20 (see Figures 1 and 2), but also image processing such as auto white balance, tone, and / or color for data obtained by capturing an image.

[0112] 7 as an example, a first trained model 106 is stored in the storage 88. Also, as described above, each time a captured image 75 is stored in the image memory 46, the training data generation unit 86A generates training data 98, and the generated training data 98 ​​is stored in the storage 88. The determination unit 86C determines whether or not the number of captured images 75 of the training data 98 ​​stored in the storage 88 has reached a first threshold value (e.g., "10,000") since the first trained model 106 was stored in the storage 88.

[0113] It should be noted that the first threshold is not a fixed value, but a variable value that changes depending on the number of captured images 75. For example, at a stage where learning processing for the CNN 104 has not been performed, the first threshold is "10000", and when the number of captured images 75 reaches "10000", the value counted as the number of captured images 75 is reset to "0", and the first threshold is set to "1000". Thereafter, each time the number of captured images 75 reaches "1000", the value counted as the number of captured images 75 is reset to "0", and the first threshold is set to "1000". Here, the values ​​"10000" and "1000" shown as the first threshold are merely examples, and other values ​​may be used.

[0114] When the determination unit 86C determines that the number of captured images 75 of the teacher data 98 ​​stored in the storage 88 has reached the first threshold, the model generation unit 86B generates a duplicate model 116. The duplicate model 116 is a learned model obtained by duplicating the first trained model 106 in the storage 88. The model generation unit 86B performs a learning process on the duplicate model 116 using the teacher data 98 ​​in the storage 88, thereby generating a second trained model 118, which is a trained model used for control related to imaging. That is, the model generation unit 86B generates the second trained model 118 by performing a learning process on the duplicate model 116 using the teacher data 98 ​​in the storage 88, on the condition that the number of captured images 75 of the teacher data 98 ​​stored in the storage 88 has reached the first threshold.

[0115] 8, the model generation unit 86B acquires training data 98 ​​from the storage 88 and inputs the captured image 75 included in the acquired training data 98 ​​to the duplicated model 116. When the captured image 75 is input, the duplicated model 116 outputs a CNN signal 116A corresponding to the various setting values ​​102. The CNN signal 116A is a signal indicating setting values ​​of the same items as the various setting values ​​102, similar to the CNN signal 104A.

[0116] The model generation unit 86B calculates an error 120 between the ground truth data 100 associated with the captured image 75 input to the replication model 116 and the CNN signal 116A. The error 120 refers to, for example, an error in the setting value of white balance R, an error in the setting value of white balance B, an error in the setting value used for exposure correction, an error in the setting value used for adjusting high color tones according to the captured scene, an error in the setting value used for adjusting shadow tones according to the captured scene, and an error in the setting value used for adjusting colors according to the captured scene.

[0117] The model generation unit 86B calculates a plurality of adjustment values ​​122 that minimize the error 120. Then, the model generation unit 86B adjusts a plurality of optimization variables in the replicated model 116 using the calculated plurality of adjustment values ​​122. Here, the plurality of optimization variables in the replicated model 116 refers to, for example, a plurality of connection weights and a plurality of offset values ​​included in the replicated model 116.

[0118] The model generation unit 86B repeats the learning process of inputting the captured images 75 to the replicated model 116, calculating the error 120, calculating the plurality of adjustment values ​​122, and adjusting the plurality of optimization variables in the replicated model 116, the number of times corresponding to the number of captured images 75 stored in the storage 88. In other words, the model generation unit 86B optimizes the replicated model 116 by adjusting the plurality of optimization variables in the replicated model 116 using the plurality of adjustment values ​​122 calculated so as to minimize the error 120 for each of the plurality of captured images 75 in the storage 88.

[0119] The model generation unit 86B optimizes the replicated model 116 to generate the second trained model 11 8. That is, the replicated model 116 is optimized by adjusting a plurality of optimization variables included in the replicated model 116, thereby generating the second trained model 118. The model generation unit 86B stores the generated second trained model 118 in the storage 88.

[0120] In the example shown in Figure 8, the teacher data 98 ​​in storage 88 is an example of "teacher data" related to the technology of the present disclosure, the captured image 75 in storage 88 is an example of "first image" related to the technology of the present disclosure, and the correct answer data 100 in storage 88 is an example of "setting value" related to the technology of the present disclosure.

[0121] 9, the storage 88 stores a reference image set 124. The reference image set 124 includes a plurality of reference images 124A. The reference images 124A are an example of the "second images" according to the technology of the present disclosure.

[0122] The multiple reference images 124A are images acquired by imaging different scenes using the imaging device 12. In the example shown in Fig. 9, as examples of the multiple reference images 124A, an image acquired by imaging a first scene using the imaging device 12, an image acquired by imaging a second scene using the imaging device 12, an image acquired by imaging a third scene using the imaging device 12, an image acquired by imaging a fourth scene using the imaging device 12, and an image acquired by imaging a fifth scene using the imaging device 12 are shown.

[0123] The execution unit 86D performs the identification process on the condition that the number of captured images 75 of the teacher data 98 ​​stored in the storage 88 has reached a first threshold. In the present embodiment, as an example, the identification process is performed depending on the output result of the second trained model 118 that has been generated on the condition that the number of captured images 75 of the teacher data 98 ​​stored in the storage 88 has reached the first threshold. This will be described in more detail below.

[0124] The execution unit 86D performs a specific process based on a first setting value 106A output from the first trained model 106 when a reference image 124A is input to the first trained model 106, and a second setting value 118A output from the second trained model 118 when the reference image 124A is input to the second trained model 118. Specifically, the execution unit 86D first inputs one reference image 124A from among the multiple reference images 124A to the first trained model 106 and the second trained model 118. When the reference image 124A is input, the first trained model 106 outputs the first setting value 106A. The first setting value 106A is a setting value for an item similar to the various setting values ​​102 (hereinafter also referred to as "various setting items"). When the reference image 124A is input, the second trained model 118 outputs the second setting value 118A. The second setting value 118A is also a setting value for various setting items.

[0125] The execution unit 86D calculates the degree of difference 125 between the first setting value 106A and the second setting value 118A. The degree of difference 125 refers to, for example, the absolute value of the difference between the first setting value 106A and the second setting value 118A. Note that the absolute value of the difference is merely one example, and the ratio of the second setting value 118A to the first setting value 106A, or the ratio of the first setting value 106A to the second setting value 118A, etc. may also be used, as long as it is a value that indicates the degree of difference between the first setting value 106A and the second setting value 118A.

[0126] Here, for example, an average value of the degrees of difference for each of the various setting items is used as the degree of difference 125 between the first setting value 106A and the second setting value 118A. For example, the degree of difference between the first setting value 106A and the second setting value 118A is an average value of the degree of difference between the first setting value 106A and the second setting value 118A for white balance R, the degree of difference between the first setting value 106A and the second setting value 118A for white balance B, the degree of difference between the first setting value 106A and the second setting value 118A used for exposure correction, the degree of difference between the first setting value 106A and the second setting value 118A used for adjusting high color tones according to the imaging scene, the degree of difference between the first setting value 106A and the second setting value 118A used for adjusting shadow tones according to the imaging scene, and the degree of difference between the first setting value 106A and the second setting value 118A used for adjusting colors according to the imaging scene. In this embodiment, the setting values ​​are converted into points, for example, so that they can be treated equally.

[0127] The execution unit 86D similarly calculates the dissimilarity 125 for the remaining reference images 124A among the plurality of reference images 124A. Then, the execution unit 86D determines whether the dissimilarity 125 for all of the plurality of reference images 124A is equal to or greater than a second threshold. If the dissimilarity 125 for all of the plurality of reference images 124A is equal to or greater than the second threshold, the execution unit 86D performs the identification process.

[0128] Here, the average of all dissimilarities is calculated, but this is merely an example. It is not necessary for there to be differences in all of the above setting values. For example, the identification process may be performed when a difference of equal to or greater than the second threshold occurs for a single setting value. The second threshold for the average value may be different from the second threshold for any single setting value. Also, while the present embodiment provides an example in which the identification process is performed on the condition that the dissimilarities 125 for all of the multiple reference images 124A are equal to or greater than the second threshold, this is merely an example. For example, the identification process may be performed on the condition that the number of times the dissimilarities 125 calculated for each of the multiple reference images 124A are determined to be equal to or greater than the second threshold is equal to or greater than a predetermined number (e.g., a majority, or any one frame) of the number of the multiple reference images 124A (in this example, five frames (frames) of the reference images 124A for the first to fifth scenes). Furthermore, for example, the identification process may be performed on the condition that the average value of the dissimilarities 125 for all of the multiple reference images 124A is equal to or greater than a second threshold. Also, for example, the identification process may be performed on the condition that the maximum value of the dissimilarities 125 for all of the multiple reference images 124A is equal to or greater than a second threshold. Also, for example, the identification process may be performed on the condition that the minimum value of the dissimilarities 125 for all of the multiple reference images 124A is equal to or greater than a second threshold. Also, for example, the identification process may be performed on the condition that the median value of the dissimilarities 125 for all of the multiple reference images 124A is equal to or greater than a second threshold.

[0129] The specific process executed by the execution unit 86D refers to a process that includes any one of the first to twelfth processes described below. However, this is merely an example, and a process that includes multiple processes among the first to twelfth processes may be executed by the execution unit 86D as the specific process.

[0130] FIG. 10 shows an example of the timing at which a series of processes (hereinafter also referred to as "verification"), including a process of calculating the dissimilarity 125 and a process of determining whether the dissimilarity 125 is equal to or greater than the second threshold, are performed by the execution unit 86D.

[0131] In the example shown in FIG. 10 , the CNN 104 is in a "learning standby" state, and no learning process is performed on the CNN 104 until the number of captured images 75 (hereinafter also referred to as the "number of captured frames") acquired by capturing images using the imaging device 12 reaches "10,000." When the number of captured frames reaches "10,000," a learning process is performed on the CNN 104, thereby generating a first trained model 106. Once the first trained model 106 is generated, operation of the first trained model 106 begins. That is, the first trained model 106 is made to perform inference (e.g., a process of inputting captured images 75, etc. to the first trained model 106, thereby outputting a first setting value 106A from the first trained model 106), and the imaging support device 14 performs processing using the inference result from the first trained model 106. An example of a process using the inference result by the first trained model 106 is a process of reflecting the first setting value 106A output from the first trained model 106 when the captured image 75 is input to the first trained model 106 in control related to imaging (e.g., a process of setting the first setting value 106A for the imaging device 12). Note that the first trained model 106 may be generated in advance by the manufacturer of the imaging device 12 and stored in a memory (e.g., NVM 64) within the camera (e.g., imaging device 12).

[0132] Furthermore, when the number of imaging frames reaches "10000", a duplicate model 116 is generated. After the duplicate model 116 is generated, the duplicate model 116 is placed in "learning standby" and no learning process is performed on the duplicate model 116 until the number of imaging frames reaches "11000".

[0133] When the number of captured frames reaches "11,000," a learning process is performed on the replicated model 116, thereby generating a second trained model 118. Once the second trained model 118 has been generated, verification is performed. In the verification, all of the reference images 124A included in the reference image set 124 are sequentially input to the first trained model 106 and the second trained model 118. As a result, a first setting value 106A is output from the first trained model 106, and a second setting value 118A is output from the second trained model 118. Then, a dissimilarity 125 between the first setting value 106A output from the first trained model 106 and the second setting value 118A output from the second trained model 118 is calculated, and it is determined whether the calculated dissimilarity 125 is greater than or equal to a second threshold. Here, if the dissimilarity 125 is greater than or equal to the second threshold, processing including a specific processing is executed.

[0134] The processing including the specific processing includes a processing for switching from the operation of the first trained model 106 to the operation of the existing second trained model 118 (e.g., the operation of the latest second trained model 118). That is, the existing second trained model 118 is made to perform inference (e.g., a processing for inputting a captured image 75, etc. to the existing second trained model 118, thereby outputting a second setting value 118A from the existing second trained model 118), and the imaging support device 14 performs processing using the inference result of the existing second trained model 118. An example of the processing using the inference result of the second trained model 118 is a processing for reflecting the second setting value 118A output from the second trained model 118 when the captured image 75 is input to the second trained model 118 in control related to imaging (e.g., a processing for setting the second setting value 118A for the imaging device 12).

[0135] On the other hand, in verification using the first trained model 106 and the existing second trained model 118, if the dissimilarity 125 is less than the second threshold, the specific processing is not executed, and the second trained model 118 is placed in "learning standby" until the number of captured frames reaches "12,000," and no learning processing is performed on the second trained model 118. Here, the existing second trained model 118 when the dissimilarity 125 is less than the second threshold, i.e., the latest second trained model 118 at the time when it is determined that the dissimilarity 125 is less than the second threshold, is an example of a "learned model" according to the technology of the present disclosure.

[0136] When the number of captured frames reaches 12,000, a learning process is performed on the existing second trained model 118, thereby generating a new second trained model 118. Once the new second trained model 118 has been generated, verification is performed. In the verification, all of the reference images 124A included in the reference image set 124 are sequentially input to the first trained model 106 and the new second trained model 118. As a result, a first setting value 106A is output from the first trained model 106, and a second setting value 118A is output from the new second trained model 118. Then, a difference 125 between the first setting value 106A output from the first trained model 106 and the second setting value 118A output from the new second trained model 118 is calculated, and it is determined whether the calculated difference 125 is greater than or equal to a second threshold. If the difference 125 is greater than or equal to the second threshold, a specific process is executed. If the dissimilarity 125 is less than the second threshold, the specific processing is not executed, and the existing second trained model 118 is placed on "learning standby" until the number of captured frames reaches "13,000," and no learning processing is performed on the existing second trained model 118. Thereafter, each time the number of captured frames reaches "1,000," the same processing is performed until the condition "dissimilarity 125 ≧ second threshold" is satisfied.

[0137] In the example shown in FIG. 10 , the time when the duplicated model 116 was duplicated, i.e., the time when the number of captured frames reached 10,000, is an example of a "specific time" according to the technology of the present disclosure. Also, in the example shown in FIG. 10 , during the period when the number of captured frames exceeds 10,000, the time when the second trained model 118 used for verification was generated is an example of a "specific time" according to the technology of the present disclosure. Also, during the period when the number of captured frames exceeds 10,000, the period from the time when the second trained model 118 used for verification was generated to the time when the number of captured frames reaches 1,000 (i.e., the period until the next verification is performed) is an example of a "period from the specific time until a condition is satisfied" according to the technology of the present disclosure.

[0138] In the example shown in FIG. 10, an example is shown in which learning processing is not performed on the CNN 104 until the number of imaging frames reaches "10000", but this is merely an example, and the number of imaging frames that is the condition for canceling learning standby may be a value less than "10000" (e.g., "1000") or a value greater than "10000" (e.g., "100000"). Also, in the example shown in FIG. 10, when the number of imaging frames exceeds "10000", verification is performed in units of "1000" images. However, this is merely an example, and the number of imaging frames that is the condition for verification may be a value less than "1000" (e.g., "100") or a value greater than "1000" (e.g., "10000").

[0139] As an example, as shown in FIG. 11, the execution unit 86D has a first processing execution unit 86D1, a second processing execution unit 86D2, a third processing execution unit 86D3, a fourth processing execution unit 86D4, a fifth processing execution unit 86D5, a sixth processing execution unit 86D6, a seventh processing execution unit 86D7, an eighth processing execution unit 86D8, a ninth processing execution unit 86D9, a tenth processing execution unit 86D10, an eleventh processing execution unit 86D11, and a twelfth processing execution unit 86D12.

[0140] The first process execution unit 86D1 executes the first process (see FIG. 12). The second process execution unit 86D2 executes the second process (see FIG. 13). The third process execution unit 86D3 executes the third process (see FIG. 14). The fourth process execution unit 86D4 executes the fourth process (see FIGS. 15 to 18). The fifth process execution unit 86D5 executes the fifth process (see FIG. 19). The sixth process execution unit 86D6 executes the sixth process (see FIG. 20). The seventh process execution unit 86D7 executes the seventh process (see FIG. 21). The eighth process execution unit 86D8 executes the eighth process (see FIG. 22). The ninth process execution unit 86D9 executes the ninth process (see FIGS. 23 and 24). The tenth process execution unit 86D10 executes the tenth process (see FIGS. 25 and 26). The eleventh process executing section 86D11 executes the eleventh process (see FIG. 27). The twelfth process executing section 86D12 executes the twelfth process (see FIG. 28).

[0141] 12, the first process executing unit 86D1 executes, as a first process, a process of reflecting the second setting value 118A in control related to imaging. For example, first, the first process executing unit 86D1 acquires a captured image 75 from the imaging device 12 and inputs the acquired captured image 75 to the second trained model 118. When the captured image 75 is input, the second trained model 118 outputs the second setting value 118A. The first process executing unit 86D1 transmits a first processing signal including the second setting value 118A output from the second trained model 118 to the imaging device 12. The imaging device 12 receives the first processing signal transmitted from the first process executing unit 86D1 and performs imaging using the second setting value 118A included in the received first processing signal.

[0142] 12 shows an example in which the captured image 75 is input to the second trained model 118, but this is merely an example, and an image other than the captured image 75 may be input to the second trained model 118. An example of an image other than the captured image 75 is at least one of the multiple reference images 124A selected by a user or the like.

[0143] 13 as an example, the second process executing unit 86D2 executes, as the second process, a process of storing the second trained model 118 in the backup storage device 94. For example, the second process executing unit 86D2 stores the latest second trained model 118 used in verification in the backup storage device 94. Note that although the backup storage device 94 is exemplified here as the storage destination for the second trained model 118, the technology of the present disclosure is not limited thereto, and instead of the backup storage device 94, a storage device of another device (e.g., the imaging device 12, a server, or a personal computer) present on the network 34 may be used.

[0144] As an example, as shown in Figure 14, the third processing execution unit 86D3 executes, as the third processing, a process of reflecting the output of either the first trained model 106 or the second trained model 118, whichever is selected in accordance with the received instruction, in control related to imaging.

[0145] In this case, for example, one of the first trained model 106 and the second trained model 118 is selected in accordance with an instruction given by a user or the like to the imaging device 12. In the example shown in Figure 14, a model instruction screen 127 is displayed on the display 28 of the imaging device 12, and an instruction from a user or the like is accepted by the touch panel 30.

[0146] The model instruction screen 127 displays a message 127A, a soft key 127B, and a soft key 127C. The message 127A is a message that prompts the user to select a trained model. In the example shown in FIG. 14, an example of the message 127A is a message that reads, "Please select a trained model." The soft key 127B is turned on via the touch panel 30 when the first trained model 106 is selected by the user or the like. The soft key 127C is turned on via the touch panel 30 when the second trained model 118 is selected by the user or the like. The example shown in FIG. 14 illustrates an example in which the soft key 127C is turned on by the user or the like. When the soft key 127B or the soft key 127C is turned on via the touch panel 30, the imaging device 12 transmits model selection information 129 to the third processing execution unit 86D3. Model selection information 129 is information indicating which of the first trained model 106 and the second trained model 118 has been selected by a user or the like (for example, information indicating which of soft keys 127B and 127C has been turned on).

[0147] The third process executing unit 86D3 receives model selection information 129 transmitted from the imaging device 12 and identifies which of the first trained model 106 and the second trained model 118 has been selected by referring to the received model selection information 129. The third process executing unit 86D3 inputs the captured image 75 to the identified one of the first trained model 106 and the second trained model 118 (hereinafter also referred to as the "selected trained model"). If the selected trained model is the first trained model 106, the first trained model 106 outputs the first setting value 106A. If the selected trained model is the second trained model 118, the second trained model 118 outputs the second setting value 118A.

[0148] The third processing execution unit 86D3 transmits a signal including the first setting value 106A output from the first trained model 106 or the second setting value 118A output from the second trained model 118 as a third processing signal to the imaging device 12. Hereinafter, for convenience of explanation, when there is no need to distinguish between the first setting value 106A output from the first trained model 106 and the second setting value 118A output from the second trained model 118, they will be referred to as "output setting values."

[0149] The imaging device 12 receives the third processing signal transmitted from the third processing execution section 86D3, and performs imaging using the output setting value included in the received third processing signal.

[0150] In the example shown in Figure 14, an example form is shown in which a captured image 75 is input to each of the first trained model 106 and the second trained model 118, but this is merely one example, and an image other than the captured image 75 may be input to at least one of the first trained model 106 and the second trained model 118.

[0151] As an example, as shown in Figure 15, as a fourth process, the fourth process execution unit 86D4 outputs data for displaying on the display 28 (see Figure 16) a simulation image 128A corresponding to an image obtained by inputting the reference image set 124 into the first trained model 106 and applying the first setting value 106A output from the first trained model 106 to the reference image set 124, and a simulation image 128B obtained by inputting the reference image set 124 into the second trained model 118 and applying the second setting value 118A output from the second trained model 118 to the reference image set 124.

[0152] Here, the reference image set 124 is an example of a "third image" and a "fifth image" according to the technology of the present disclosure. That is, the "third image" and the "fifth image" may be the same image. Furthermore, the first setting value 106A is an example of a "first output result" according to the technology of the present disclosure. Furthermore, the second setting value 118A is an example of a "second output result" according to the technology of the present disclosure. Furthermore, the simulation image set 126B is an example of a "fourth image" according to the technology of the present disclosure. Furthermore, the simulation image 128B is an example of a "sixth image" according to the technology of the present disclosure. Furthermore, the display 28 is an example of a "first display" according to the technology of the present disclosure.

[0153] In the example shown in FIG. 15 , the fourth process executing unit 86D4 sequentially inputs each of all reference images 124A included in the reference image set 124 to the first trained model 106. The first trained model 106 outputs a first setting value 106A each time a reference image 124A is input. Each time a first setting value 106A is output from the first trained model 106, the fourth process executing unit 86D4 generates a simulation image set 126A including a simulation image 128A and a usage model identifier 130A based on the output first setting value 106A and the corresponding reference image 124A. The simulation image set 126A is generated for each of all reference images 124A included in the reference image set 124. Note that, although all reference images 124A are sequentially input to the first trained model 106 here, one or more of the reference images 124A may be input.

[0154] The simulation image 128A is an image predicted as the captured image 75 to be acquired by the imaging device 12 when imaging is performed using the first setting value 106A. An example of the simulation image 128A is an image corresponding to the reference image 124A that is affected by the first setting value 106A when imaging is performed using the first setting value 106A, on the premise that the reference image 124A is acquired by imaging performed by the imaging device 12.

[0155] The usage model identifier 130A included in the simulation image set 126A is an identifier capable of identifying the first trained model 106. The usage model identifier 130A is associated with the simulation image 128A.

[0156] In the example shown in FIG. 15 , the fourth process executing unit 86D4 sequentially inputs each of all reference images 124A included in the reference image set 124 to the second trained model 118. The second trained model 118 outputs a second setting value 118A each time a reference image 124A is input. Each time a second setting value 118A is output from the second trained model 118, the fourth process executing unit 86D4 generates a simulation image set 126B including a simulation image 128B and a usage model identifier 130B based on the output second setting value 118A and the corresponding reference image 124A. The simulation image set 126B is generated for each of all reference images 124A included in the reference image set 124.

[0157] The simulation image 128B is an image predicted as the captured image 75 acquired by the imaging device 12 when imaging is performed using the second setting value 118A. An example of the simulation image 128B is an image corresponding to the reference image 124A that is affected by the second setting value 118A when imaging is performed using the second setting value 118A, on the premise that the reference image 124A is acquired by imaging using the imaging device 12.

[0158] The usage model identifier 130B included in the simulation image set 126B is an identifier capable of identifying the second trained model 118. The usage model identifier 130B is associated with the simulation image set 126B.

[0159] For ease of explanation, data including simulation image sets 126A and 126B will be referred to as simulation image set 126. Furthermore, when it is not necessary to distinguish between simulation images 128A and 128B, they will be referred to as simulation image 128. Furthermore, when it is not necessary to distinguish between usage model identifiers 130A and 130B, they will be referred to as usage model identifier 130.

[0160] The fourth process executing unit 86D4 transmits the simulation image set 126 to the imaging device 12. The imaging device 12 receives the simulation image set 126 transmitted from the fourth process executing unit 86D4. Here, the simulation image set 126 is an example of "first data" according to the technology of the present disclosure.

[0161] 16, the CPU 62 of the imaging device 12 generates a simulation image display screen 132 based on the simulation image set 126, and causes the display 28 to display the simulation image display screen 132. The simulation image display screen 132 displays a simulation image 128 and the like.

[0162] Simulation image display screen 132 includes first screen 132A and second screen 132B. CPU 62 generates first screen 132A based on simulation image set 126A, and generates second screen 132B based on simulation image set 126B. In the example shown in Fig. 16, the upper half of simulation image display screen 132 is first screen 132A, and the lower half is second screen 132B.

[0163] CPU 62 references usage model identifier 130 and controls display 28 so that simulation image 128A is displayed on first screen 132A and simulation image 128B is displayed on second screen 132B. Note that there may be one or more simulation images 128A and 128B displayed. As a result, simulation image 128A and simulation image 128B are displayed on display 28 in a manner that allows them to be distinguished. Note that usage model identifier 130 is an example of "data for displaying the fourth image and the sixth image on the first display in a manner that allows them to be distinguished" according to the technology of the present disclosure.

[0164] CPU 62 displays message 132A1 on display 28 in association with simulation image 128A based on usage model identifier 130A included in simulation image set 126A. CPU 62 also displays message 132B1 on display 28 in association with simulation image 128B based on usage model identifier 130B included in simulation image set 126B. In the example shown in Fig. 16 , message 132A1 is displayed on first screen 132A, thereby associating message 132A1 with simulation image 128A, and message 132B1 is displayed on second screen 132B, thereby associating message 132B1 with simulation image 128B.

[0165] Message 132A1 is a message that can identify the first trained model 106, and message 132B1 is a message that can identify the second trained model 118. In the example shown in Figure 16, an example of message 132A1 is a message that reads "Simulation image using the first trained model," and an example of message 132B1 is a message that reads "Simulation image using the second trained model."

[0166] 16 is merely an example, and message 132A1 may be any message indicating that simulation image 128A displayed on first screen 132A is a simulation image generated based on first trained model 106. Furthermore, the content of message 132B1 illustrated in FIG. 16 is merely an example, and message 132B1 may be any message indicating that simulation image 128B displayed on second screen 132B is a simulation image generated based on second trained model 118.

[0167] Message 132A1 is an example of "first trained model identifying information" according to the technology of the present disclosure, and message 132B1 is an example of "second trained model identifying information" according to the technology of the present disclosure. Furthermore, usage model identifier 130 is an example of "data for displaying on the first display a fourth image in association with first trained model identifying information capable of identifying the first trained model, and for displaying on the first display a sixth image in association with second trained model identifying information capable of identifying the second trained model" according to the technology of the present disclosure.

[0168] Here, message 132A1 identifies that simulation image 128A is an image based on first trained model 106, and message 132B1 identifies that simulation image 128B is an image based on second trained model 118, but the technology of the present disclosure is not limited to this. For example, the outer frame of simulation image 128A may be colored to identify it as a simulation image based on first trained model 106, and the outer frame of simulation image 128B may be colored to identify it as a simulation image based on second trained model 118. Furthermore, a mark or the like that identifies it as a simulation image based on the first trained model 106 may be displayed in association with simulation image 128A, and a mark or the like that identifies it as a simulation image based on the second trained model 118 may be displayed in association with simulation image 128B.

[0169] 17 , when a simulation image display screen 132 is displayed on the display 28 and a user or the like selects one of the simulation images 128 in the simulation image display screen 132 via the touch panel 30, the CPU 62 transmits selected image specification information 134, which can specify which simulation image 128 was selected, to the imaging support device 14. The fourth process execution unit 86D4 of the imaging support device 14 receives the selected image specification information 134 transmitted from the imaging device 12.

[0170] As an example, as shown in FIG. 18, as one of the processes included in the fourth process, the fourth process execution unit 86D4 executes a process in which, when simulation image 128A is selected from among multiple simulation images 128 displayed on the display 28, the output of the first trained model 106 is reflected in the control related to imaging, and when simulation image 128B is selected, the output of the second trained model 118 is reflected in the control related to imaging.

[0171] In this case, for example, the fourth process executing unit 86D4 refers to the selected image identification information 134 to identify which of the multiple simulation images 128 has been selected by a user or the like. The fourth process executing unit 86D4 inputs the reference image 124A used to generate the identified simulation image 128 into the trained model used to generate the identified simulation image 128. The trained model used to generate the simulation image 128 is the first trained model 106 or the second trained model 118. When the reference image 124A is input to the first trained model 106, the first trained model 106 outputs the first setting value 106A, and when the reference image 124A is input to the second trained model 118, the second trained model 118 outputs the second setting value 118A.

[0172] The fourth process execution unit 86D4 transmits a signal including the output setting value as a fourth processing signal to the imaging device 12. The imaging device 12 receives the fourth processing signal transmitted from the fourth process execution unit 86D4, and performs imaging using the output setting value included in the received fourth processing signal.

[0173] As an example, as shown in FIG. 19 , the fifth process executing unit 86D5 executes, as the fifth process, a process of outputting data for displaying, on the display 28, time identification information 136 that can identify the time when the second trained model 118 was generated. Note that multiple pieces of time identification information 136 may be displayed on the display 28 without displaying an image. For example, information on multiple dates identified by the multiple pieces of time identification information 136 may be listed and displayed on the display 28. Furthermore, the user may be able to select the trained model to use by selecting one of the dates.

[0174] In this case, for example, the fifth process executing unit 86D5 determines whether the second trained model 118 has been generated by the model generating unit 86B (see FIG. 8). When the second trained model 118 has been generated by the model generating unit 86B, the fifth process executing unit 86D5 acquires the time identification information 136. For example, the time identification information 136 is acquired from a clock (for example, a real-time clock). The fifth process executing unit 86D5 acquires the latest second setting value 118A. An example of the latest second setting value 118A is the second setting value 118A used to calculate the dissimilarity 125 when the condition "dissimilarity 125≧second threshold" is satisfied.

[0175] The fifth process executing unit 86D5 generates a second setting value reflected image 138. The second setting value reflected image 138 is an image obtained by reflecting the output of the second trained model 118, i.e., the latest second setting value 118A, in the reference image 124A. In other words, the second setting value reflected image 138 is generated based on the latest second setting value 118A and the reference image 124A in a manner similar to the manner in which the fourth process executing unit 86D4 generated the simulation image 128 ( FIG. 15 ).

[0176] The fifth process executing unit 86D5 associates the second setting value reflected image 138 with the time identification information 136. Then, the fifth process executing unit 86D5 transmits the second setting value reflected image 138 associated with the time identification information 136 to the imaging device 12. The imaging device 12 receives the second setting value reflected image 138 associated with the time identification information 136. Under the control of the CPU 62, the imaging device 12 displays the second setting value reflected image 138 and the time identified from the time identification information 136 (for example, the time when the second trained model 118 was generated) side by side on the display 28.

[0177] The time-identifying information 136 and the second setting value-reflecting image 138 are "second data" according to the technology of the present disclosure. The time-identifying information 136 and the second setting value-reflecting image 138 are an example of "data for displaying the time-identifying information on the second display in association with a seventh image reflecting the output of the second trained model" according to the technology of the present disclosure. The second setting value-reflecting image 138 is an example of a "seventh image" according to the technology of the present disclosure.

[0178] As an example, as shown in FIG. 20 , the sixth process executing unit 86D6 executes, as the sixth process, a process of associating the time identification information 136 with the second trained model 118. In this case, for example, when the second trained model 118 is generated by the model generating unit 86B, the sixth process executing unit 86D6 acquires the time identification information 136. The sixth process executing unit 86D6 stores the acquired time identification information 136 in the storage 88 in association with the second trained model 118 in the storage 88. Note that in the example shown in FIG. 20 , the second trained model 118 and the time identification information 136 are stored in the storage 88 in an associated state, but the technology of the present disclosure is not limited to this. The second trained model 118 and the time identification information 136 may also be stored in the backup storage device 94 in an associated state.

[0179] In addition, the CPU 86 of the imaging support device 14 may perform a learning process on the duplicated trained model by treating the existing second trained model 118 (in the example shown in Figure 20, the second trained model 118 stored in the storage 88) in the same way as the first trained model 106, and treating the duplicated trained model, which is a trained model duplicated from the existing second trained model 118, in the same way as the duplicated model 116.

[0180] In this case, the CPU 86 generates a third trained model by performing a learning process on the duplicated trained model, and associates the generated third trained model with the time identification information 136, as with the second trained model 118. Then, the CPU 86 stores the third trained model and the time identification information 136 in an associated state in the storage 88. By repeatedly performing a process similar to that of generating the third trained model by performing a learning process on the second trained model 118, multiple operational trained models are accumulated in the storage 88 in an associated state with the time identification information 136. In other words, if N is a natural number greater than or equal to 3, N or more trained models are accumulated in the storage 88 in an associated state with the time identification information 136. Note that the N or more trained models may be accumulated in the backup storage device 94 in an associated state with the time identification information 136.

[0181] As an example, as shown in FIG. 21, the seventh process executing unit 86D7 executes, as the seventh process, a process of reflecting the output of the second trained model 118 in control related to imaging at a predetermined timing.

[0182] In this case, for example, the seventh process executing unit 86D7 stores the latest second setting value 118A output from the second trained model 118 in the storage 88. The latest second setting value 118A may be, for example, the second setting value 118A used in the latest verification. The seventh process executing unit 86D7 determines whether a predetermined timing has arrived since the latest second setting value 118A was stored in the storage 88. Examples of predetermined timings include the timing when the imaging device 12 is started up, the timing when the number of captured images 75 obtained by imaging by the imaging device 12 (for example, the number of captured images 75 obtained by imaging by the imaging device 12 since the latest second setting value 118A was stored in the storage 88) becomes equal to or greater than a sixth threshold (for example, "10,000"), the timing when the operating mode of the imaging device 12 transitions from playback mode to setting mode, or the timing when a rating is given to the captured image 75 in playback mode (for example, an evaluation by a user or the like of the image quality of the captured image 75).

[0183] When a predetermined timing arrives, the seventh process executing unit 86D7 acquires the second setting value 118A from the storage 88 and reflects the acquired second setting value 118A in control related to imaging. In this case, for example, the seventh process executing unit 86D7 transmits a signal including the second setting value 118A acquired from the storage 88 as a seventh processing signal to the imaging device 12. The imaging device 12 receives the seventh processing signal transmitted from the seventh process executing unit 86D7 and performs imaging using the second setting value 118A included in the received seventh processing signal.

[0184] As an example, as shown in FIG. 22, when the eighth process execution unit 86D8 applies the second trained model 118 to another device 140, which is an imaging device different from the imaging device 12, as the eighth process, it executes a process to correct at least one of the data input to the second trained model 118 and the output from the second trained model 118 based on the characteristics of the imaging device 12 and the characteristics of the other device 140.

[0185] 22, the storage 88 stores the second trained model 118 and image sensor information 142. The image sensor information 142 is information relating to a plurality of image sensors. The image sensor information 142 is associated with the second trained model 118 and includes characteristic information 142A and individual difference information 142B.

[0186] The characteristic information 142A is information indicating the characteristics of each of the different image sensors involved in the second trained model 118. Examples of the different image sensors involved in the second trained model 118 include the image sensor 20 of the imaging device 12 to which the second trained model 118 is currently applied (i.e., the imaging device 12 to which the output of the second trained model 118 is currently reflected), and an image sensor mounted on a new object to which the second trained model 118 is applied. In the example shown in FIG. 22 , the image sensor mounted on the new object to which the second trained model 118 is applied is image sensor 140A mounted on another device 140.

[0187] The individual difference information 142B is information indicating individual differences between different image sensors involved in the second trained model 118. Examples of individual differences include the difference between the sensitivity of each RGB pixel of one image sensor (e.g., image sensor 20) and the sensitivity of each RGB pixel of the other image sensor (e.g., an image sensor mounted on another device 140), the difference between the ISO sensitivity of one image sensor and the ISO sensitivity of the other image sensor, and / or the difference between the electronic shutter speed of one image sensor and the electronic shutter speed of the other, second image sensor.

[0188] The eighth process executing unit 86D8 acquires, from the separate device 140, separate device information 144 including information capable of identifying the image sensor 140A as information indicating the characteristics of the separate device 140. The eighth process executing unit 86D8 uses the separate device information 144 and the image sensor information 142 to identify the characteristics of the imaging device 12 (e.g., the characteristics identifiable from the image sensor information 142) and the characteristics of the separate device 140 (e.g., the characteristics identifiable from the image sensor information 142). For example, the eighth process executing unit 86D8 acquires at least one of characteristic information 142A related to the image sensor 140A identified from the separate device information 144 and individual difference information 142B related to the image sensor 140A identified from the separate device information 144, and identifies the characteristics of the imaging device 12 and the characteristics of the separate device 140 using at least one of the characteristic information 142A and the individual difference information 142B.

[0189] The second trained model 118 is a model obtained by performing a learning process using multiple captured images 75 acquired by capturing images with the imaging device 12 and the correct answer data 100. Therefore, if this is applied as is to the other device 140, the characteristics of the imaging device 12 will be reflected in the second setting value 118A. Therefore, the eighth process executing unit 86D8 corrects the reference image set 124 used as input for the second trained model 118 and the second setting value 118A output from the second trained model 118 based on the characteristics of the imaging device 12 and the characteristics of the other device 140.

[0190] In this case, for example, the eighth processing execution unit 86D8 derives the content of the correction to be performed on the reference image set 124 using a first table (not shown) or a first arithmetic formula (not shown) in which the difference between the characteristics of the imaging device 12 and the characteristics of the separate device 140 is associated with the content of the correction to be performed on the reference image set 124, and corrects the reference image set 124 in accordance with the derived content of the correction.

[0191] The eighth process executing unit 86D8 inputs the corrected reference image set 124 to the second trained model 118. When the corrected reference image set 124 is input, the second trained model 118 outputs a second setting value 118A. The eighth process executing unit 86D8 derives the content of the correction to be made to the second setting value 118A output from the second trained model 118 using a second table (not shown) or a second arithmetic formula (not shown) in which the difference between the characteristics of the imaging device 12 and the characteristics of the separate device 140 is associated with the content of the correction to be made to the second setting value 118A, and corrects the second setting value 118A in accordance with the derived content of the correction.

[0192] The eighth process executing unit 86D8 reflects the corrected second setting value 118A in control related to imaging. In this case, for example, the eighth process executing unit 86D8 transmits a signal including the corrected second setting value 118A as an eighth processing signal to the imaging device 12. The imaging device 12 receives the eighth processing signal transmitted from the eighth process executing unit 86D8 and performs imaging using the corrected second setting value 118A included in the received eighth processing signal.

[0193] Note that, while correction is performed on both the reference image set 124 and the second setting value 118A here, the technology of the present disclosure is not limited thereto, and correction may be performed on only one of the reference image set 124 and the second setting value 118A. Also, while an example in which the reference image set 124 is input to the second trained model 118 is given here, the technology of the present disclosure is not limited thereto, and at least one reference image 124A or at least one captured image 75 may be input to the second trained model 118. Furthermore, in a manner similar to the manner in which correction is performed on the reference image set 124, the eighth process executing unit 86D8 may correct the captured image 75 obtained by capturing an image with the imaging device 12 based on the characteristics of the imaging device 12 and the characteristics of the separate device 140. Further, here, the image sensor information 142 is exemplified as information including both characteristic information 142A and individual difference information 142B, but the technology of the present disclosure is not limited to this, and the image sensor information 142 may be information including only one of characteristic information 142A and individual difference information 142B.

[0194] As an example, as shown in FIG. 23, the ninth process execution unit 86D9 executes, as the ninth process, a process of outputting a simulation image set 126B as data for displaying the above-described simulation image 128B and an unprocessed image 146A on the display 28 (see FIG. 24). The unprocessed image 146A is an image obtained by inputting a reference image 124A included in the reference image set 124 into the second trained model 118 without applying the second setting value 118A output from the second trained model 118 to the reference image 124A (for example, an image equivalent to the reference image 124A itself included in the reference image set 124). In other words, the unprocessed image 146A is an image that reflects only image processing that does not use the output results of the trained model.

[0195] In the example shown in FIG. 23, reference image 124A is an example of an "eighth image" according to the technology of the present disclosure. Second setting value 118A is an example of a "third output result" according to the technology of the present disclosure. Simulation image 128B is an example of a "first processed image" according to the technology of the present disclosure. Unprocessed image 146A is an example of an "unprocessed image" according to the technology of the present disclosure. Simulation image set 126 and unprocessed image set 146 are examples of "third data" according to the technology of the present disclosure.

[0196] The ninth process executing unit 86D9 generates a simulation image set 126B in a manner similar to the processing performed by the fourth process executing unit 86D4 shown in Fig. 15, and transmits the generated simulation image set 126B to the imaging device 12. The ninth process executing unit 86D9 also transmits the reference image set 124 used in the input to the second trained model 118 to the imaging device 12 as an unprocessed image set 146. The imaging device 12 receives the simulation image set 126B and the unprocessed image set 146 transmitted from the ninth process executing unit 86D9.

[0197] 24, the CPU 62 of the imaging device 12 generates a simulation image display screen 148 based on the simulation image set 126B and the unprocessed image set 146, and causes the display 28 to display the simulation image display screen 148. The simulation image display screen 148 displays the simulation image 128B, the unprocessed image 146A, and the like.

[0198] Simulation image display screen 148 includes first screen 148A and second screen 148B. CPU 62 generates first screen 148A based on unprocessed image set 146, and generates second screen 148B based on simulation image set 126B. In the example shown in Fig. 24, the upper half of simulation image display screen 148 is first screen 148A, and the lower half is second screen 148B.

[0199] CPU 62 controls display 28 so that unprocessed image 146A is displayed on first screen 148A and simulation image 128B is displayed on second screen 148B. As a result, unprocessed image 146A and simulation image 128B are displayed on display 28 in a manner that allows them to be distinguished from each other.

[0200] CPU 62 causes message 148A1 to be displayed on display 28 in association with unprocessed image 146A. CPU 62 also causes message 148B1 to be displayed on display 28 in association with simulation image 128B. In the example shown in Fig. 24, message 148A1 is displayed on first screen 148A, thereby causing message 148A1 to be associated with unprocessed image 146A, and message 148B1 is displayed on second screen 148B, thereby causing message 148B1 to be associated with simulation image 128B.

[0201] Message 148A1 is a message capable of identifying unprocessed image 146A, and message 148B1 is a message capable of identifying second trained model 118. In the example shown in Figure 24, an example of message 148A1 is a message "unprocessed image," and an example of message 148B1 is a message "simulation image using second trained model."

[0202] 24 is merely an example, and message 148A1 may be any message indicating that unprocessed image 146A displayed on first screen 148A is not an image generated based on a trained model. Also, the content of message 148B1 illustrated in FIG. 24 is merely an example, and message 148B1 may be any message indicating that simulation image 128B displayed on second screen 148B is a simulation image generated based on second trained model 118.

[0203] Here, message 148A1 identifies that unprocessed image 146A is not an image generated based on a trained model, and message 148B1 identifies that simulation image 128B is an image generated based on second trained model 118; however, the technology of the present disclosure is not limited to this. For example, the outer frame of unprocessed image 146A may be colored to identify that unprocessed image 146A is not an image generated based on a trained model, and the outer frame of simulation image 128B may be colored to identify that it is a simulation image generated based on second trained model 118. Furthermore, a mark or the like that identifies that unprocessed image 146A is not an image generated based on a trained model may be displayed in association with unprocessed image 146A, and a mark or the like that identifies that it is a simulation image generated based on second trained model 118 may be displayed in association with simulation image 128B.

[0204] 23 and 24 show an example in which unprocessed image 146A is reference image 124A and simulation image 128B is an image generated based on second setting value 118A output from second trained model 118 when reference image 124A is input to second trained model 118, but the technology of the present disclosure is not limited to this. For example, captured image 75 may be used instead of unprocessed image 146A, and an image generated based on second setting value 118A output from second trained model 118 when captured image 75 is input to second trained model 118 may be used instead of simulation image 128B.

[0205] As an example, as shown in Figure 25, the 10th process execution unit 86D10 executes, as the 10th process, a process of outputting data for displaying on the display 28 (see Figure 26) the above-mentioned unprocessed image 146A and a brightness-adjusted image 150A in which the brightness of the simulation image 128B has been adjusted based on the second setting value 118A output from the second trained model 118.

[0206] In the example shown in FIG. 25, unprocessed image 146A is an example of a "ninth image" according to the technology of the present disclosure. Second set value 118A is an example of a "fourth output result" according to the technology of the present disclosure. Brightness-adjusted image 150A is an example of a "second processed image" according to the technology of the present disclosure. Unprocessed image 146A and brightness-adjusted image set 150 are an example of "fourth data" according to the technology of the present disclosure.

[0207] The tenth process executing unit 86D10 generates a simulation image set 126B based on the second setting value 118A in a manner similar to the processing performed by the fourth process executing unit 86D4 shown in FIG. 15 , and further generates a brightness-adjusted image set 150 by adjusting the brightness of the simulation image 128B based on the second setting value 118A. The brightness-adjusted image set 150 includes a brightness-adjusted image 150A and a usage model identifier 150B. The tenth process executing unit 86D10 generates the brightness-adjusted image 150A by adjusting the brightness of the simulation image 128B included in the simulation image set 126B. The tenth process executing unit 86D10 associates the usage model identifier 150B with the brightness-adjusted image 150A. The usage model identifier 150B is an identifier corresponding to the usage model identifier 130B associated with the simulation image 128B before the brightness was adjusted (for example, an identifier obtained by duplicating the usage model identifier 130B).

[0208] The tenth process execution unit 86D10 transmits the unprocessed image 146A and the brightness-adjusted image set 150 to the imaging device 12. The imaging device 12 receives the unprocessed image 146A and the brightness-adjusted image set 150 transmitted from the tenth process execution unit 86D10.

[0209] 26, the CPU 62 of the imaging device 12 generates a simulation image display screen 152 based on the unprocessed image 146A and the brightness-adjusted image set 150, and causes the display 28 to display the simulation image display screen 152. The simulation image display screen 152 displays the unprocessed image 146A, the brightness-adjusted image 150A, etc.

[0210] The simulation image display screen 152 includes a first screen 152A and a second screen 152B. The CPU 62 generates the first screen 152A based on the unprocessed image 146A, and generates the second screen 152B based on the brightness-adjusted image set 150. In the example shown in Fig. 26, the upper half of the simulation image display screen 152 is the first screen 152A, and the lower half is the second screen 152B.

[0211] CPU 62 controls display 28 so that unprocessed image 146A is displayed on first screen 152A and brightness-adjusted image 150A is displayed on second screen 152B. As a result, unprocessed image 146A and brightness-adjusted image 150A are displayed on display 28 in a manner that allows them to be distinguished from each other.

[0212] CPU 62 displays message 152A1 on display 28 in association with unprocessed image 146A. CPU 62 also displays message 152B1 on display 28 in association with brightness-adjusted image 150A. In the example shown in Fig. 26, message 152A1 is displayed on first screen 152A, thereby causing message 152A1 to be associated with unprocessed image 146A, and message 152B1 is displayed on second screen 152B, thereby causing message 152B1 to be associated with brightness-adjusted image 150A.

[0213] Message 152A1 is a message that can identify unprocessed image 146A, and message 152B1 is a message that can identify that brightness adjustment has been performed on simulation image 128B. In the example shown in Fig. 26, an example of message 152A1 is a message that reads "Unprocessed image," and an example of message 152B1 is a message that reads "Image obtained by adjusting brightness on a simulation image using the second trained model."

[0214] 26 is merely an example, and message 152A1 may be any message indicating that unprocessed image 146A displayed on first screen 152A is an image that has not been subjected to processing based on a trained model. Also, the content of message 152B1 illustrated in FIG. 26 is merely an example, and message 152B1 may be any message that can identify that brightness-adjusted image 150A displayed on second screen 152B is an image whose brightness has been adjusted based on the output of second trained model 118.

[0215] Here, message 152A1 identifies that unprocessed image 146A is an image that does not rely on second trained model 118, and message 152B1 identifies that brightness-adjusted image 150A is an image in which brightness has been adjusted for simulation image 128B based on the output of second trained model 118, but the technology of the present disclosure is not limited to this. For example, the outer frame of unprocessed image 146A may be colored to identify unprocessed image 146A as an image that does not rely on second trained model 118, and the outer frame of brightness-adjusted image 150A may be colored to identify brightness-adjusted image 150A as an image in which brightness has been adjusted based on the output of second trained model 118.

[0216] In addition, a mark or the like that can identify that unprocessed image 146A is an image that does not rely on second trained model 118 may be displayed in association with unprocessed image 146A, and a mark or the like that can identify that brightness-adjusted image 150A is an image whose brightness has been adjusted based on the output of second trained model 118 may be displayed in association with brightness-adjusted image 150A.

[0217] In the examples shown in FIGS. 25 and 26, the brightness-adjusted image 150A is output from the second trained model 118 when the reference image 124A is input to the second trained model 118. Although an example in which an image is generated based on the second set value 118A is shown, the technology of the present disclosure is not limited to this. For example, instead of the reference image 124A, an image generated based on the second set value 118A output from the second trained model 118 when the captured image 75 is input to the second trained model 118 may be used.

[0218] 27, the eleventh process executing unit 86D11 inputs a reference image 124A included in the reference image set 124 to the second trained model 118. When the reference image 124A is input, the second trained model 118 outputs a second setting value 118A.

[0219] The eleventh process executing unit 86D11 reflects the second setting value 118A in control related to imaging in a manner similar to the process performed by the first process executing unit 86D1 shown in Fig. 12, the process performed by the third process executing unit 86D3 shown in Fig. 14, the process performed by the seventh process executing unit 86D7 shown in Fig. 21, and the process performed by the eighth process executing unit 86D8 shown in Fig. 22. In this manner, a captured image 75 acquired by imaging by the imaging device 12 with the second setting value 118A reflected in control related to imaging is stored in the image memory 46. In the example shown in Fig. 27, the captured image 75 stored in the image memory 46 is an example of a "third processed image" according to the technology of the present disclosure.

[0220] The CPU 62 of the imaging device 12 acquires a captured image 75 from the image memory 46 and adds accompanying information 154 associated with the captured image 75 to the acquired captured image 75. The accompanying information 154 includes, for example, the second setting value 118A used in capturing the captured image 75, information indicating the characteristics of the imaging device 12, the capturing conditions, and the capture date and time. The accompanying information 154 may be, for example, Exif information. Note that in the example shown in FIG. 27 , the accompanying information 154 is an example of “first accompanying information” according to the technology of the present disclosure.

[0221] The CPU 62 of the imaging device 12 transmits the captured image 75 with the accompanying information 154 added to the eleventh process executing unit 86D11. The eleventh process executing unit 86D11 receives the captured image 75 transmitted from the CPU 62 of the imaging device 12. As the eleventh process, the eleventh process executing unit 86D11 executes a process of including the above-mentioned usage model identifier 130B in the accompanying information 154 added to the received captured image 75.

[0222] In the example shown in Figure 27, an example is shown in which the reference image 124A is input to the second trained model 118, but this is merely one example, and the captured image 75 may also be input to the second trained model 118.

[0223] 28, the twelfth process executing unit 86D12 inputs a reference image 124A included in the reference image set 124 to the first trained model 106. When the reference image 124A is input, the first trained model 106 outputs a first setting value 106A.

[0224] The twelfth process executing unit 86D12 reflects the first setting value 106A in control related to imaging in a manner similar to the processing performed by the first process executing unit 86D1 shown in FIG. 12, the processing performed by the third process executing unit 86D3 shown in FIG. 14, the processing performed by the seventh process executing unit 86D7 shown in FIG. 21, the processing performed by the eighth process executing unit 86D8 shown in FIG. 22, and the processing performed by the eleventh process executing unit 86D11 shown in FIG. 27. In this manner, a captured image 75 acquired by imaging performed by the imaging device 12 with the first setting value 106A reflected in control related to imaging is stored in the image memory 46. Note that in the example shown in FIG. 28, the captured image 75 stored in the image memory 46 is an example of a “fourth processed image” according to the technology of the present disclosure.

[0225] The CPU 62 of the imaging device 12 acquires a captured image 75 from the image memory 46 and adds accompanying information 154 associated with the captured image 75 to the acquired captured image 75. The accompanying information 154 includes, for example, the first setting value 106A used in capturing the captured image 75, information indicating the characteristics of the imaging device 12, the capturing conditions, and the capture date and time. The accompanying information 154 may be, for example, Exif information. In the example shown in FIG. 28 , the accompanying information 154 is an example of “second accompanying information” according to the technology of the present disclosure.

[0226] The CPU 62 of the imaging device 12 transmits the captured image 75 with the accompanying information 154 added to the twelfth process executing unit 86D12. The twelfth process executing unit 86D12 receives the captured image 75 transmitted from the CPU 62 of the imaging device 12. As the twelfth process, the twelfth process executing unit 86D12 executes a process of including the above-mentioned usage model identifier 130A in the accompanying information 154 added to the received captured image 75.

[0227] Next, the operation of the imaging system 10 will be described with reference to FIGS. 29A and 29B.

[0228] 29A and 29B show an example of the flow of imaging support processing performed by the CPU 86 of the imaging support device 14. Note that the flow of imaging support processing shown in Fig. 29A and 29B is an example of an "imaging support method" according to the technique of the present disclosure.

[0229] 29A, first, in step ST100, the teacher data generation unit 86A determines whether or not the captured image 75 has been stored in the image memory 46 of the imaging device 12. If the captured image 75 has not been stored in the image memory 46 of the imaging device 12 in step ST100, the determination is negative, and the determination of step ST100 is made again. If the captured image 75 has been stored in the image memory 46 of the imaging device 12 in step ST100, the determination is positive, and the imaging support process proceeds to step ST102.

[0230] In step ST102, the teacher data generating unit 86A acquires the captured image 75 from the image memory 46. After the processing of step ST102 is executed, the imaging support processing proceeds to step ST104.

[0231] In step ST104, the teacher data generating unit 86A acquires various setting values ​​102 from the NVM 64 of the imaging device 12. After the processing of step ST104 is executed, the imaging support processing proceeds to step ST106.

[0232] In step ST106, the teacher data generation unit 86A generates teacher data 98 ​​based on the captured image 75 acquired in step ST102 and the various setting values ​​102 acquired in step ST104, and stores the generated teacher data 98 ​​in the storage 88. After the processing of step ST106 is executed, the imaging support processing proceeds to step ST108.

[0233] In step ST108, the determination unit 86C determines whether or not a first learning process execution timing has arrived, which is the timing to execute a learning process on the CNN 104. An example of the first learning process execution timing is the timing when the number of imaging frames reaches a first threshold value (for example, "10000").

[0234] In step ST108, if the timing to execute the first learning process has not arrived, the determination is negative, and the imaging support process proceeds to step ST100. In step ST108, if the timing to execute the first learning process has arrived, the determination is positive, and the imaging support process proceeds to step ST110.

[0235] In step ST110, the model generation unit 86B generates the first trained model 106 by performing a training process on the CNN 104 using the training data 98 ​​stored in the storage 88. After the processing of step ST110 is executed, the imaging support processing proceeds to step ST112.

[0236] In step ST112, the determination unit 86C determines whether or not the number of imaging frames has reached the first threshold. If the number of imaging frames has not reached the first threshold in step ST112, the determination is negative, and the determination of step ST112 is made again. If the number of imaging frames has reached the first threshold in step ST112, the determination is positive, and the imaging support processing proceeds to step ST114.

[0237] In step ST114, the model generation unit 86B generates a duplicate model 116 from the first trained model 106 generated in step ST110. After the processing of step ST114 is executed, the imaging support processing proceeds to step ST116.

[0238] In step ST116, the teacher data generation unit 86A determines whether or not the captured image 75 has been stored in the image memory 46 of the imaging device 12. If the captured image 75 has not been stored in the image memory 46 of the imaging device 12 in step ST116, the determination is negative, and the determination of step ST116 is made again. If the captured image 75 has been stored in the image memory 46 of the imaging device 12 in step ST116, the determination is positive, and the imaging support processing proceeds to step ST118.

[0239] In step ST118, the teacher data generating unit 86A acquires the captured image 75 from the image memory 46. After the processing of step ST118 is executed, the imaging support processing proceeds to step ST120.

[0240] In step ST120, the teacher data generating unit 86A acquires various setting values ​​102 from the NVM 64 of the imaging device 12. After the processing of step ST120 is executed, the imaging support processing proceeds to step ST122.

[0241] In step ST122, the teacher data generation unit 86A generates teacher data 98 ​​based on the captured image 75 acquired in step ST118 and the various setting values ​​102 acquired in step ST120, and stores the generated teacher data 98 ​​in the storage 88. After the processing of step ST122 is executed, the imaging support processing proceeds to step ST124.

[0242] In step ST124, the determination unit 86C determines whether or not the second learning process execution timing has arrived, which is the timing to execute the learning process on the replicated model 116 generated in step ST114. An example of the second learning process execution timing is the timing when the number of captured frames reaches a first threshold value (for example, "1000").

[0243] In step ST124, if the timing to execute the second learning process has not arrived, the determination is negative, and the imaging support process proceeds to step ST116. In step ST124, if the timing to execute the second learning process has arrived, the determination is positive, and the imaging support process proceeds to step ST126 shown in FIG. 29B.

[0244] In step ST126 shown in FIG. 29B, the model generation unit 86B uses the training data 98 ​​stored in the storage 88 (for example, the training data 98 ​​obtained by repeatedly performing the processes of steps ST116 to ST124) to perform a training process on the latest model (for example, the duplicated model 116 when a first positive judgment is made in step ST124, and the existing second trained model 118 (i.e., the latest trained model 118) when a second or subsequent positive judgment is made in step ST124), thereby generating the second trained model 118. After the process of step ST126 is performed, the imaging support process proceeds to step ST128.

[0245] In step ST128, the execution unit 86D acquires the reference image set 124 from the storage 88. After the processing of step ST128 is executed, the imaging support processing proceeds to step ST130.

[0246] In step ST130, the execution unit 86D inputs the reference image set 124 acquired in step ST128 into the first trained model 106 and the second trained model 118. After the processing of step ST130 is executed, the imaging support processing proceeds to step ST132.

[0247] In step ST132, the execution unit 86D acquires the first setting value 106A output from the first trained model 106 and the second setting value 118A output from the second trained model 118. After the processing of step ST132 is executed, the imaging support processing proceeds to step ST134.

[0248] In step ST134, the execution unit 86D calculates the degree of difference 125 between the first setting value 106A and the second setting value 118A acquired in step ST132. After the processing of step ST134 is executed, the imaging support processing proceeds to step ST136.

[0249] In step ST136, the execution unit 86D determines whether the dissimilarity 125 calculated in step ST134 is equal to or greater than the second threshold. If the dissimilarity 125 calculated in step ST134 is less than the second threshold, the determination is negative, and the imaging support processing proceeds to step ST116 shown in Fig. 29A. If the dissimilarity 125 calculated in step ST134 is equal to or greater than the second threshold, the determination is positive, and the imaging support processing proceeds to step ST138.

[0250] In step ST138, the execution section 86D executes the specific processing. After the processing in step ST138 is executed, the imaging support processing ends.

[0251] As described above, in the imaging support device 14, the first trained model 106 is stored in the storage 88, and the first trained model 106 is used for control related to imaging. Furthermore, in the imaging support device 14, a learning process is performed on the learned model using the various setting values ​​102 applied to the imaging device 12 when the captured image 75 was acquired as the correct answer data 100, and the captured image 75 and the correct answer data 100 as the teacher data 98, thereby generating a second trained model 118 used for control related to imaging. Then, a specification process is performed based on the first setting value 106A output from the first trained model 106 when the reference image set 124 is input to the first trained model 106, and the second setting value 118A output from the second trained model 118 when the reference image set 124 is input to the second trained model 118. Therefore, this configuration can contribute to reducing the load on the CPU 86 and / or the user compared to when only processing unrelated to the degree of difference between the first trained model 106 and the second trained model 118 is performed.

[0252] In the imaging support device 14, the reference image set 124 is stored in the storage 88. Therefore, according to this configuration, the process of inputting the reference image set 124 to the first trained model 106 and the second trained model 118 can be easily realized compared to when the images to be input to the first trained model 106 and the second trained model 118 are not stored in any memory such as the storage 88.

[0253] In the imaging support device 14, when the condition that the number of imaging frames has reached the first threshold is satisfied, a learning process is performed to generate the second trained model 118. Therefore, with this configuration, the load on the learning process can be reduced compared to when the learning process is constantly performed.

[0254] In the imaging support device 14, the multiple captured images 75 acquired by imaging with the imaging device 12 during the period from the time when the latest learning model to be subjected to the learning process was obtained (for example, the time when the replication model 116 was generated) until the condition that the number of captured frames has reached the first threshold is satisfied, and the various setting values ​​related to the multiple captured images 75 and applied to the imaging device 12 are used as teacher data 98. Therefore, with this configuration, the load on the CPU 86 for the learning process can be reduced compared to when the learning process is performed using a single captured image 75 and a single setting value as teacher data each time imaging is performed.

[0255] In the imaging support device 14, the specific processing is performed when the condition that the number of imaging frames has reached the first threshold is satisfied. Therefore, with this configuration, the load on the CPU 86 can be reduced compared to when the specific processing is always performed.

[0256] In the imaging support device 14, the identification process is performed when the difference 125 between the first setting value 106A and the second setting value 118A is equal to or greater than the second threshold value. Therefore, with this configuration, the load on the CPU 86 can be reduced compared to when the identification process is always performed regardless of the difference 125 between the first setting value 106A and the second setting value 118A.

[0257] In the imaging support device 14, a process of reflecting the second setting value 118A in control related to imaging is performed as a first process included in the specific process. Therefore, according to this configuration, the control related to imaging can be made closer to the control intended by the user, etc., compared to a case where only the first trained model 106 is always used for control related to imaging.

[0258] In the imaging support device 14, as a second process included in the specific process, a process is performed in which the second trained model 118 is stored in the backup storage device 94. Therefore, according to this configuration, the same second trained model 118 can be used repeatedly.

[0259] In the imaging support device 14, as a third process included in the specific process, a process is performed in which the output of either the first trained model 106 or the second trained model 118, whichever is selected in accordance with the instruction received by the third process execution unit 86D3, is reflected in control related to imaging. Therefore, with this configuration, the output of either the first trained model 106 or the second trained model 118, whichever corresponds to the preference of the user, etc., can be reflected in control related to imaging.

[0260] In the imaging support device 14, as a fourth process included in the specific process, data for displaying on the display 28 a simulation image 128A corresponding to an image obtained by inputting the reference image set 124 to the first trained model 106 and applying the first setting value 106A output from the first trained model 106 to the reference image set 124, and a simulation image 128B obtained by inputting the reference image set 124 to the second trained model 118 and applying the second setting value 118A output from the second trained model 118 to the reference image set 124, is transmitted to the imaging device 12. Therefore, with this configuration, the difference between the output of the first trained model 106 and the output of the second trained model 118 can be visually recognized by a user or the like.

[0261] In the imaging support device 14, a simulation image 128A corresponding to an image obtained by inputting the reference image set 124 into the first trained model 106 and applying the first setting value 106A output from the first trained model 106 to the reference image set 124, and a simulation image 128B obtained by inputting the reference image set 124 into the second trained model 118 and applying the second setting value 118A output from the second trained model 118 to the reference image set 124 are displayed on the display 28 in a distinguishable manner. Therefore, with this configuration, a user or the like can easily perceive the difference between the simulation image 128A and the simulation image 128B, compared to when the simulation image 128A and the simulation image 128B are displayed on the display 28 in an indistinguishable manner.

[0262] In the imaging support device 14, the simulation image 128A and the message 132A1 are displayed in association with each other on the first screen 132A, and the simulation image 128B and the message 132B1 are displayed in association with each other on the second screen 132B. Therefore, according to this configuration, the user can easily perceive that the simulation image 128A is an image obtained by using the first trained model 106, and that the simulation image 128B is an image obtained by using the second trained model 118.

[0263] In the imaging support device 14, as a fourth process included in the specific process, data for displaying on the display 28 a simulation image 128A corresponding to an image obtained by inputting the reference image set 124 to the first trained model 106 and applying the first setting value 106A output from the first trained model 106 to the reference image set 124, and a simulation image 128B obtained by inputting the reference image set 124 to the second trained model 118 and applying the second setting value 118A output from the second trained model 118 to the reference image set 124, is transmitted to the imaging device 12. Therefore, with this configuration, it is easier to reflect the output of a trained model intended by a user, etc., from the first trained model 106 and the second trained model 118, in control related to imaging, compared to when the output of a trained model randomly selected from the first trained model 106 and the second trained model 118 is reflected in control related to imaging.

[0264] In the imaging support device 14, as a fifth process included in the identification process, data for displaying, on the display 28, time identification information 136 that can identify the time when the second trained model 118 was generated is transmitted to the imaging device 12. Therefore, according to this configuration, it is possible for a user or the like to perceive the time when the second trained model 118 was generated.

[0265] In the imaging support device 14, the second setting value reflected image 138 reflecting the output of the second trained model 118 is displayed on the display 28 in a correlated state with the time identification information 136. Therefore, according to this configuration, the user or the like can perceive the correspondence between the time when the second trained model 118 was generated and the image reflecting the output of the second trained model 118.

[0266] In the imaging support device 14, as a sixth process included in the identification process, a process of associating the time identification information 136 with the second trained model 118 is performed. Therefore, according to this configuration, it is possible to allow a user or the like to perceive the correspondence between the time when the second trained model 118 was generated and the second trained model 118.

[0267] In the imaging support device 14, as a seventh process included in the specific process, a process is performed in which the output of the second trained model 118 is reflected in control related to imaging at a predetermined timing. Therefore, with this configuration, it is possible to update the trained model at a timing that is convenient for the user, compared to, for example, updating the trained model at the timing when a significant difference is determined.

[0268] In the imaging support device 14, as a seventh process included in the specific process, a process is performed to reflect the output of the second trained model 118 in control related to imaging when the imaging device 12 is started up, when the number of captured images 75 acquired by imaging by the imaging device 12 (for example, the number of captured images 75 acquired by imaging by the imaging device 12 since the latest second setting value 118A was stored in the storage 88) becomes equal to or greater than a sixth threshold (for example, "10,000"), when the operation mode of the imaging device 12 transitions from playback mode to setting mode, or when a rating is given to the captured image 75 in playback mode (for example, an evaluation by a user or the like of the image quality of the captured image 75). Therefore, with this configuration, the trained model can be updated at a time that is convenient for the user, compared to, for example, updating the trained model when a significant difference is determined.

[0269] In the imaging support device 14, as an eighth process included in the specific process, when the second trained model 118 is applied to another device 140, which is an imaging device different from the imaging device 12, a process is performed to correct at least one of the data input to the second trained model 118 and the output from the second trained model 118 based on the characteristics of the imaging device 12 and the characteristics of the other device 140. Therefore, according to this configuration, compared to when the second trained model 118 is applied to another device without considering the characteristics of the imaging device 12 and the characteristics of the other device 140 at all, the image quality of the image obtained by imaging by the other device 140 can be easily made closer to the image quality intended by the user, etc.

[0270] In the imaging support device 14, the second trained model 118 is accompanied by image sensor information 142, which includes characteristic information 142A and individual difference information 142B. The characteristics of the imaging device 12 and the characteristics of the separate device 140 are identified based on the image sensor information 142. Therefore, according to this configuration, the image quality intended by the user or the like can be more accurately reproduced for images obtained by imaging using the separate device 140, compared to when the second trained model 118 is applied to the separate device 140 without taking into consideration any differences between the image sensor 20 used in the imaging device 12 and the image sensor 140A used in the separate device 140.

[0271] In the imaging support device 14, as a ninth process included in the specific process, a simulation image 128B obtained by inputting the reference image set 124 to the second trained model 118 and applying the second setting value 118A output from the second trained model 118 to the reference image set 124, and an unprocessed image 146A are displayed on the display 28. The unprocessed image 146A is an image obtained by inputting the reference image 124A included in the reference image set 124 to the second trained model 118 without applying the second setting value 118A output from the second trained model 118 to the reference image 124A (e.g., an image equivalent to the reference image 124A itself included in the reference image set 124). Therefore, this configuration allows a user or the like to perceive the difference between an image influenced by the output of the second trained model 118 and an image not influenced by the output of the second trained model 118.

[0272] In the imaging support device 14, as a tenth process included in the specific process, the reference image set 124 is input to the second trained model 118, and the second setting value 118A output from the second trained model 118 is applied to the reference image set 124 to adjust the brightness, thereby transmitting data for displaying the brightness-adjusted image 150A and the unprocessed image 146A on the display 28 to the imaging device 12. Therefore, with this configuration, the user or the like can perceive the difference between the unprocessed image and the image in which the brightness of the image influenced by the output of the second trained model 118 has been adjusted.

[0273] In the imaging support device 14, as an eleventh process included in the identification process, a process is performed in which the usage model identifier 130B is included in the accompanying information 154 added to the captured image 75. Therefore, with this configuration, it is easier to identify that an image obtained by imaging while reflecting the output of the second trained model 118 on controls related to imaging is an image obtained using the second trained model 118, compared to a case in which information capable of identifying the second trained model 118 is not associated with an image obtained by imaging while reflecting the output of the second trained model 118 on controls related to imaging.

[0274] In the imaging support device 14, as a twelfth process included in the identification process, a process is executed in which the used model identifier 130A is included in the accompanying information 154 added to the captured image 75. Therefore, with this configuration, it is easier to identify that an image obtained by capturing an image while reflecting the output of the first trained model 106 on controls related to imaging is an image obtained using the first trained model 106, compared to a case in which information capable of identifying the first trained model 106 is not associated with an image obtained by capturing an image while reflecting the output of the first trained model 106 on controls related to imaging.

[0275] In the imaging support device 14, various setting values ​​102 are used as training data 98. A setting value related to the white balance used in imaging, a setting value related to the exposure used in imaging, a setting value related to the saturation used in imaging, and a setting value related to the gradation used in imaging are adopted as the various setting values ​​102. Therefore, according to this configuration, compared to a case where setting values ​​completely unrelated to the setting values ​​related to the white balance used in imaging, the setting value related to the exposure used in imaging, the setting value related to the saturation used in imaging, and the setting value related to the gradation used in imaging are used, it is possible to more easily bring at least one of the setting values ​​related to the white balance used in imaging, the setting value related to the exposure used in imaging, the setting value related to the saturation used in imaging, and the setting value related to the gradation used in imaging closer to the setting value intended by the user, etc.

[0276] Furthermore, a setting value related to the focus used in imaging may be used as the teacher data 98. Examples of the setting value related to the focus used in imaging include a setting value related to a focus frame used for AF control and / or a setting value related to the AF method to be used (for example, a phase difference AF method or a contrast AF method). In this case, compared to using a setting value completely unrelated to the setting value related to the focus used in imaging as the teacher data 98, it is possible to more easily bring the setting value related to the focus used in imaging closer to the setting value intended by the user or the like.

[0277] The setting value 102 used as the teacher data 98 ​​may be at least one of a setting value related to the white balance used in imaging, a setting value related to the exposure used in imaging, a setting value related to the saturation used in imaging, a setting value related to the gradation used in imaging, and a setting value related to the focus used in imaging.

[0278] Note that, in the above embodiment, an example form (see FIG. 10 ) in which verification is performed by inputting the reference image set 124 into the first trained model 106 and the second trained model 118 has been described, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 30 , verification may be performed by inputting 1,000 captured images 75 into the first trained model 106 and the second trained model 118. In this case, as shown in FIG. 31 as an example, each time a captured image 75 is stored in the image memory 46 of the imaging device 12, the captured image 75 is also stored in the storage 88. Then, when the number of captured images 75 stored in the storage 88 reaches 1,000, the execution unit 86D inputs the 1,000 captured images 75 in the storage 88 into the first trained model 106 and the second trained model 118. As a result, the first trained model 106 outputs a first setting value 106A corresponding to the input captured image 75, and the second trained model 118 outputs a second setting value 118A corresponding to the input captured image 75. The execution unit 86D calculates the degree of difference 125 between the first setting value 106A and the second setting value 118A, and determines whether the degree of difference 125 is greater than or equal to a second threshold value.

[0279] In this case, for example, if a second trained model 118 is generated by performing a learning process using 1,000 captured images 75, and one of the 1,000 captured images 75 used in the learning process to generate the second trained model 118 is given to the generated second trained model 118 and the first trained model 106 as input for verification, the trained model used for operation may be switched to the latest trained model, or a specific process may be performed, provided that the number of images at which the difference 125 between the first set value 106A and the second set value 118A reaches the second threshold reaches a specified number (e.g., 800).

[0280] Furthermore, the embodiment in which verification is not performed until 1,000 captured images 75 have been accumulated is merely one example, and verification may be performed each time one capture (one image) is taken, and when the number of images at which the difference 125 between the first set value 106A and the second set value 118A reaches the second threshold reaches a specified number, the trained model used for operation may be switched to the latest trained model, or specific processing may be performed.

[0281] Note that, here, an example is given in which 1,000 captured images 75 are input to the first trained model 106 and the second trained model 118, but this is merely one example, and it is possible to input fewer than 1,000 captured images 75 (e.g., 100) or more than 1,000 (e.g., 10,000) captured images 75 to the first trained model 106 and the second trained model 118.

[0282] In the above embodiment, an example of the specific processing is performed when the dissimilarity 125 is equal to or greater than the second threshold has been described. However, the technology of the present disclosure is not limited thereto. For example, as shown in FIG. 32 , the CPU 86 may perform a predetermined processing on the condition that the number of images captured within a predetermined period of time reaches a third threshold. Examples of the third threshold include 5,000 images after the replication model 116 is generated and 5,000 images after verification. However, the third threshold is not limited thereto and may be a value corresponding to a number less than 5,000 or a number greater than 5,000. The predetermined period may be, for example, one day. That is, for example, it may be determined whether 5,000 or more images have been captured in one day. If 5,000 images have been captured in one day, the second trained model 118 is considered to have a significantly greater difference than the first trained model 106.

[0283] Examples of the predetermined processing include a processing for storing the second trained model 118, when it is determined that a significant difference has occurred between the output of the first trained model 106 and the output of the second trained model 118, in the backup storage device 94, and / or a processing for determining that a significant difference has occurred between the output of the first trained model 106 and the output of the second trained model 118. Here, a significant difference refers, for example, to the degree to which the tendency of the content of the captured images 75 used in the training process of the second trained model 118 deviates, by a predetermined degree or more, from the tendency of the content (e.g., captured scene) of the captured images 75 used in the training process of the first trained model 106. Furthermore, a significant difference refers, for example, to the degree to which the tendency of the content of the supervised data 100 used in the training process of the second trained model 118 deviates, by a predetermined degree or more, from the tendency of the content of the supervised data 100 used in the training process of the first trained model 106. The predetermined degree may be a fixed value, or may be a variable value that is changed according to an instruction given by a user or the like to the imaging support device 14 and / or various conditions. Note that although the predetermined process is described here, it does not have to be predetermined, and for example, the user may be allowed to select one of these processes on the condition that the number of images captured within a predetermined period of time reaches a third threshold.

[0284] According to a configuration in which predetermined processing is performed on the condition that the number of images obtained by capturing reaches the third threshold, the load on the CPU 86 can be reduced compared to when predetermined processing is performed regardless of the number of images obtained by capturing. Furthermore, for example, if processing is performed to store in the backup storage device 94 second trained models 118 for which it is determined that a significant difference has occurred on the condition that the number of images obtained by capturing reaches the third threshold, it is possible to store in the backup storage device 94 only second trained models 118 that are considered to have a particularly large difference, compared to when all second trained models 118 for which it is determined that a significant difference has occurred are stored in the backup storage device 94.

[0285] Furthermore, the CPU 86 may perform a predetermined process when the number of captured images 75 used as the teacher data 98 ​​obtained by capturing images in a first environment during a predetermined period is equal to or greater than a fourth threshold, and the number of captured images 75 used as the teacher data 98 ​​obtained by capturing images in a second environment different from the first environment is equal to or less than a fifth threshold. The predetermined period refers to, for example, one day. For example, when the fourth threshold is significantly greater than the fifth threshold (for example, the fourth threshold is 1000 and the fifth threshold is 10), and the number of captured images 75 satisfies the above condition, it is considered that the captured images 75 include many images captured in the first environment. In other words, the captured images 75 that form the basis of the teacher data 98 ​​are captured in the first environment. It is considered that the second trained model 118 was obtained by capturing images disproportionately downward. If training is performed using training data 98 ​​that was captured in the first environment, it is considered that the second trained model 118 will have a significantly greater difference than the first trained model 106. This specifying process may also include a process of storing the second trained model 118 in the backup storage device 94 and / or a process of determining that a significant difference has occurred between the output of the first trained model 106 and the output of the second trained model 118. These processes also do not need to be determined in advance. For example, the user may be allowed to select one of these processes when the number of captured images 75 obtained by capturing images in the first environment and used as training data 98 ​​is equal to or greater than a fourth threshold, and the number of captured images 75 obtained by capturing images in a second environment different from the first environment and used as training data 98 ​​is equal to or less than a fifth threshold. Furthermore, it is not necessary to use the number of captured images 75 obtained under the second environment. For example, it is possible to determine whether the ratio of the number of captured images 75 obtained by capturing images under the first environment to the total number of captured images 75 obtained in one day is equal to or greater than a predetermined threshold value, and then perform the above-mentioned predetermined processing.

[0286] 33, the CPU 86, assuming that the accompanying information 154 added to the captured images 75 includes information capable of identifying the first environment and the second environment, references the accompanying information 154 for each of the 10,000th to 11,000th captured images 75 to identify the number of captured images 75 acquired by capturing images in the first environment (hereinafter referred to as the "number of frames in the first environment") and the number of captured images 75 acquired by capturing images in the second environment (hereinafter referred to as the "number of frames in the second environment"). Here, the first environment and the second environment refer to, for example, environments identified from the captured scene and the light source. The captured scene refers to, for example, a person or a landscape, and the light source refers to, for example, the sun or an indoor light source.

[0287] The CPU 86 determines whether the condition (hereinafter also referred to as the "determination condition") that the number of frames under the first environment is equal to or greater than a fourth threshold and the number of frames under the second environment is equal to or less than a fifth threshold is satisfied. Satisfaction of the determination condition means that a significant difference has occurred between the output of the first trained model 106 and the output of the second trained model 118. Therefore, when the determination condition is satisfied, the CPU 86 executes a predetermined process. Note that the fourth and fifth thresholds may be fixed values ​​or may be variable values ​​that are changed according to instructions given to the imaging support device 14 by a user or the like and / or various conditions.

[0288] In this way, the CPU 86 identifies the number of frames under the first environment and the number of frames under the second environment by referring to the accompanying information 154, and determines whether the determination condition is satisfied based on the identification result. Therefore, with this configuration, the load on the CPU 86 can be reduced from the time the CPU 86 acquires the captured image 75 until the predetermined processing is performed, compared to the case where the captured image 75 is input to the first trained model 106 and the second trained model 118, the first setting value 106A is output from the first trained model 106, the second setting value 118A is output from the second trained model 118, and the dissimilarity 125 between the first setting value 106A and the second setting value 118A is calculated. In addition, by determining whether or not images 75 used as training data 98 ​​were taken disproportionately under the first environment, and if images 75 used as training data 98 ​​were taken disproportionately under the first environment, a process is performed to store in the backup storage device 94 the second trained models 118 that are determined to have had a significant difference.Compared to storing all second trained models 118 that are determined to have had a significant difference in the backup storage device 94, only the second trained models 118 that are considered to have a particularly large difference can be stored in the backup storage device 94.

[0289] 33 shows an example in which the CPU 86 identifies the number of frames under the first environment and the number of frames under the second environment by referring to the accompanying information 154 of the captured image 75, but this is merely one example, and for example, as shown in Fig. 34, the CPU 86 may identify the number of frames under the first environment and the number of frames under the second environment by performing object recognition processing on each captured image 75. The object recognition processing may be an AI-based object recognition processing or a template matching-based object recognition processing.

[0290] Although the above embodiment has been described assuming that one type of interchangeable lens 18 is attached to the imaging device body 16, the technology of the present disclosure is not limited thereto. For example, as shown in FIG. 35 , the imaging device 12 is a lens-interchangeable imaging device, and different types of interchangeable lenses 18 are selectively attached to the imaging device body 16. Therefore, the CPU 86 of the imaging support device 14 generates multiple second trained models 118 by performing a learning process on a learning model (e.g., a replica model 116) for each type of interchangeable lens 18 used to capture the captured image 75 included in the training data 98. The CPU 86 stores the generated multiple second trained models 118 in the storage 88. Therefore, with this configuration, the output of the second trained model 118 suitable for the interchangeable lens 18 attached to the imaging device body 16 can be reflected in the control related to imaging, compared to a case where the output from only one second trained model 118 is always reflected in the control related to imaging, regardless of the type of interchangeable lens 18.

[0291] 36, when an interchangeable lens 18 is attached to the imaging device body 16, the CPU 86 acquires the second trained model 118 corresponding to the interchangeable lens 18 attached to the imaging device body 16 from among the multiple second trained models 118 stored in the storage 88. The second trained model 118 corresponding to the interchangeable lens 18 attached to the imaging device body 16 refers to the second trained model 118 generated by using, as training data 98, a captured image 75 acquired by capturing an image with the imaging device 12 to which the interchangeable lens 18 is attached. The CPU 86 performs various processes (e.g., a process of reflecting the output of the second trained model 118 in control related to imaging, and / or a learning process for the second trained model 118) using the second trained model 118 acquired from the storage 88. Therefore, with this configuration, it is possible to perform processing using a second trained model 118 that is suitable for the interchangeable lens 18, compared to performing processing using one randomly selected from among a plurality of second trained models 118.

[0292] In the examples shown in Figures 35 and 36, an example is given in which, when a different type of interchangeable lens 18 is selectively attached to the imaging device body 16, a second trained model 118 corresponding to the interchangeable lens 18 attached to the imaging device body 16 is used.However, in a similar manner, when multiple imaging systems, each having the function of imaging a subject, are selectively used, a second trained model 118 corresponding to the selected imaging system may be used.

[0293] 37 , if the smart device 155 has imaging systems 155A and 155B as multiple imaging systems, and each of the first imaging system 155A and the second imaging system 155B is used to capture an image 75 included in the training data 98, the CPU 86 of the imaging support device 14 generates multiple second trained models 118 for each of the first imaging system 155A and the second imaging system 155B by performing a learning process on a learning model (e.g., a replicated model 116, etc.). The CPU 86 stores the generated multiple second trained models 118 in the storage 88. Therefore, according to this configuration, compared to when the output from only one second trained model 118 is always reflected in the control related to imaging by the first imaging system 155A and the second imaging system 155B of the smart device 155, the output of the second trained model 118 suitable for whichever of the first imaging system 155A and the second imaging system 155B is used can be reflected in the control related to imaging by whichever of the first imaging system 155A and the second imaging system 155B is used. Note that the smart device 155 is an example of an "imaging device" according to the technology of the present disclosure, and the first imaging system 155A and the second imaging system 155B are examples of "multiple imaging systems" according to the technology of the present disclosure.

[0294] 38, when an imaging system to be used for imaging is selected from first imaging system 155A and second imaging system 155B, CPU 86 acquires, from among the multiple second trained models 118 in storage 88, a second trained model 118 corresponding to the selected one of first imaging system 155A and second imaging system 155B. The second trained model 118 corresponding to the selected one of first imaging system 155A and second imaging system 155B refers to the second trained model 118 generated by using, as training data 98, a captured image 75 acquired by imaging by the selected one of first imaging system 155A and second imaging system 155B. The CPU 86 performs various processes (e.g., a process of reflecting the output of the second trained model 118 in control of imaging by a selected one of the first imaging system 155A and the second imaging system 155B, and / or a learning process for the second trained model 118) using the second trained model 118 obtained from the storage 88. Therefore, with this configuration, it is possible to perform processing using a second trained model 118 that is suitable for a selected one of the first imaging system 155A and the second imaging system 155B, out of the multiple second trained models 118, compared to a case where processing is performed using one randomly selected from the multiple second trained models 118.

[0295] FIG. 39 shows a configuration example of a smart device 156 equipped with multiple imaging systems with a magnification change function. As shown in FIG. 39 as an example, the smart device 156 includes a first imaging system 156A and a second imaging system 156B. Each of the first imaging system 156A and the second imaging system 156B is generally also referred to as an out-camera. Each of the first imaging systems 156A and 156B has a magnification change function with a different magnification change range. Also, similar to the examples shown in FIGS. 37 and 38, a second trained model 118 is assigned to each of the first imaging system 156A and the second imaging system 156B.

[0296] As shown in FIG. 40A as an example, the smart device 156 includes a touch panel display 158. The touch panel display 158 has a display 160 and a touch panel 162. When a captured image 164 is displayed on the display 160 as a live view image and an instruction to change the angle of view is given to the touch panel 162, a zoom function is activated to change the angle of view. In the example shown in FIG. 40A, a pinch-out operation is shown as the instruction to change the angle of view. When a pinch-out operation is performed on the touch panel 162 while the captured image 164 is displayed on the display 160, the zoom function is activated to enlarge the captured image 164 within the display 160. Note that the method of zooming is not limited to this, and as shown in FIG. 40B as an example, a zoom ratio may be directly selected using soft keys 160A, 160B, and 160C. 40B, softkey 160A is a softkey that is turned on when selecting a magnification of 1, softkey 160B is a softkey that is turned on when selecting a magnification of 0.5, and softkey 160C is a softkey that is turned on when selecting a magnification of 2.5. Note that the magnifications illustrated here are merely examples, and other magnifications may be used. Furthermore, the number of softkeys is not limited to three, namely softkeys 160A, 160B, and 160C, and may be two, or four or more.

[0297] 41 as an example, the smart device 156 transmits used imaging system information 166 capable of identifying which of the first imaging system 156A and the second imaging system 156B is currently being used, and a captured image 164 obtained by capturing an image using the currently being used imaging system, to the imaging support device 14. The captured image 164 is classified into a captured image 164A obtained by capturing an image using the first imaging system 156A and a captured image 164B obtained by capturing an image using the second imaging system 156B.

[0298] The imaging support device 14 has a second trained model 118 for the first imaging system 156A and a second trained model 118 for the second imaging system 156B, and the second trained model 118 for the first imaging system 156A is a model obtained by performing a learning process using, as teacher data, multiple captured images 164A and setting values ​​applied to the first imaging system 156A in imaging to obtain the multiple captured images 164A. The second trained model 118 for the second imaging system 156B is a model obtained by performing a learning process using, as teacher data, multiple captured images 164B and setting values ​​applied to the second imaging system 156B in imaging to obtain the multiple captured images 164B.

[0299] The CPU 86 of the imaging support device 14 receives the used imaging system information 166 and the captured image 164 transmitted from the smart device 156. The CPU 86 inputs the received captured image 164 to the second trained model 118 corresponding to the imaging system identified from the received used imaging system information 166. For example, if the imaging system identified from the received used imaging system information 166 is the first imaging system 156A, the CPU 86 inputs the captured image 164A to the second trained model 118 for the first imaging system 156A. Furthermore, if the imaging system identified from the received used imaging system information 166 is the second imaging system 156B, the CPU 86 inputs the captured image 164B to the second trained model 118 for the second imaging system 156B.

[0300] When captured image 164A is input, second trained model 118 for first imaging system 156A outputs second setting value 118A. CPU 86 reflects second setting value 118A output from second trained model 118 for first imaging system 156A in control related to imaging by first imaging system 156A. Furthermore, when captured image 164B is input, second trained model 118 for second imaging system 156B outputs second setting value 118A. CPU 86 reflects second setting value 118A output from second trained model 118 for second imaging system 156B in control related to imaging by second imaging system 156B.

[0301] Incidentally, methods for switching the imaging system used for imaging from one of first imaging system 156A and second imaging system 156B to the other include a method of switching each time at least one soft key (not shown) displayed on display 160 is turned on, and a method of stepless switching by giving stepless instructions such as a pinch-in operation and a pinch-out operation to touch panel 162. In the stepless switching method, CPU 86 changes the angle of view steplessly in response to stepless instructions such as a pinch-in operation and a pinch-out operation, and may switch from one of first imaging system 156A and second imaging system 156B to the other while the angle of view is being changed. In other words, when the angle of view is changed within a range where the magnification range of the first imaging system 156A and the magnification range of the second imaging system 156B overlap (hereinafter also referred to as the "overlapping magnification range"), or within a range where the magnification range of the first imaging system 156A and the magnification range of the second imaging system 156B do not overlap (hereinafter also referred to as the "non-overlapping magnification range"), the first imaging system 156A and the second imaging system 156B are not switched from one to the other, but when the angle of view is changed across the overlapping magnification range and the non-overlapping magnification range, the first imaging system 156A and the second imaging system 156B are switched from one to the other.

[0302] The output of the second trained model 118 for the first imaging system 156A is reflected in the control related to imaging by the first imaging system 156A, and the output of the second trained model 118 for the second imaging system 156B is reflected in the control related to imaging by the second imaging system 156B. Therefore, as shown in Figure 42 as an example, if switching is made from one of the first imaging system 156A and the second imaging system 156B to the other while the angle of view is being changed, hunting may occur in the captured image 164 due to the difference between the second trained model 118 for the first imaging system 156A and the second trained model 118 for the second imaging system 156B.

[0303] 43, the CPU 86 of the imaging support device 14 receives a stepless instruction (hereinafter referred to as a "angle of view change instruction") received via the touch panel 162 of the smart device 156 as an instruction to change the angle of view. If the received angle of view change instruction involves switching the imaging system midway, the CPU 86 continues to use the second trained model 118 assigned to the imaging system before the switch in the imaging system after the switch. This prevents hunting caused by differences between the second trained models 118 from occurring at a timing unintended by the user, even when switching from one of the first imaging system 156A and the second imaging system 156B to the other.

[0304] Smart device 156 is an example of an "imaging device" according to the technology of the present disclosure, and first imaging system 156A and second imaging system 156B are an example of a "plurality of imaging systems" according to the technology of the present disclosure.

[0305] Incidentally, when the smart device 156 is started up (hereinafter also referred to as the "start-up timing"), if the imaging system that is frequently used by the user, etc., is started up out of the first imaging system 156A and the second imaging system 156B, it becomes possible to start imaging quickly without switching the imaging system.

[0306] Therefore, as an example, as shown in FIG. 44, various setting values ​​102 used as the supervised answer data 100 include information related to the imaging system as the setting values. The information related to the imaging system refers to, for example, information related to the imaging system selected when the captured image 75 was acquired by the smart device 156 (for example, information that can identify the imaging system). By including the information related to the imaging system in the supervised answer data 100 in this way, the model generation unit 86B can make the duplicated model 116 learn the tendency of imaging systems that are frequently used in the smart device 156. By making the duplicated model 116 learn the tendency of imaging systems that are frequently used in the smart device 156, the duplicated model 116 is optimized, and a second trained model 118 is generated.

[0307] 45, the CPU 86 inputs a captured image 164 obtained by selectively capturing images using the first imaging system 156A and the second imaging system 156B of the smart device 156 to a second trained model 118 obtained by optimizing the replicated model 116 by learning the tendency of imaging systems frequently used by the smart device 156 for a certain imaging scene. When the captured image 164 is input, the second trained model 118 outputs a second setting value 118A. The CPU 86 sets the imaging system to be used at the startup timing of the smart device 156 by reflecting the second setting value 118A output from the second trained model 118 in control related to imaging by the smart device 156, and causes the smart device 156 to selectively use the first imaging system 156A and the second imaging system 156B at the startup timing of the smart device 156. This allows the user to quickly use the imaging system they desire when the smart device 156 is started up, compared to when an imaging system randomly selected from the first imaging system 156A and the second imaging system 156B is used when the smart device 156 is started up.

[0308] When the output of the second trained model 118 obtained by performing a learning process using information including information related to the imaging system as correct answer data 100 is reflected in control related to imaging by the smart device 156, as shown in FIG. 46 as an example, the CPU 168 of the smart device 156 sets the position of the angle of view to be used at the startup timing of the smart device 156 within the overlapping zoom range (for example, the center of the overlapping zoom range) so that imaging is started with an angle of view within the overlapping zoom range for the imaging system used at the startup timing of the smart device 156.

[0309] The information related to the imaging system included in the supervised answer data 100 may be information including history information of angle of view change instructions. If the second trained model 118 is generated by performing a learning process using teacher data 98 ​​including supervised answer data 100 configured in this manner, the output of the second trained model 118 can be reflected in control related to imaging by the smart device 156, allowing the CPU 168 of the smart device 156 to set the position of the angle of view to a frequently used position within the magnification range.

[0310] 37 and 38 illustrate two imaging systems, first imaging system 155A and second imaging system 155B, and the example shown in Fig. 39 illustrates two imaging systems, first imaging system 156A and second imaging system 156B, but the technology of the present disclosure is not limited to this and may include three or more imaging systems. Furthermore, the device equipped with multiple imaging systems is not limited to smart devices 155 and 156 and may be, for example, an imaging device (e.g., a surveillance camera) that captures light of multiple wavelength bands using different imaging systems.

[0311] In the above embodiment, an example has been described in which the CPU 86 performs the identification process based on the first setting value 106A and the second setting value 118A, but the technology of the present disclosure is not limited to this. For example, the CPU 86 may perform the identification process based on the degree of difference between the first trained model 106 and the second trained model 118. For example, the CPU 86 performs the identification process when the degree of difference between the first trained model 106 and the second trained model 118 is equal to or greater than a default difference. The default difference may be a fixed value, or may be a variable value that changes depending on instructions given to the imaging support device 14 by a user or the like and / or various conditions.

[0312] Furthermore, each of the first trained model 106 and the second trained model 118 may be a model having an input layer, multiple intermediate layers, and an output layer, and the CPU 86 may perform the identification process based on the degree of difference of at least one layer (e.g., at least one specified layer) between the first trained model 106 and the second trained model 118. In this case, the at least one layer may be, for example, multiple intermediate layers and an output layer, all of the multiple intermediate layers, some of the multiple intermediate layers (e.g., at least one specified layer), or the output layer.

[0313] In the above embodiment, an example in which the imaging device 12 and the imaging support device 14 are separate has been described, but the technology of the present disclosure is not limited to this, and the imaging device 12 and the imaging support device 14 may be integrated. In this case, for example, as shown in Fig. 47, an imaging support processing program 96 may be stored in the NVM 64 of the imaging device main body 16, and the CPU 62 may execute the imaging support processing program 96.

[0314] Furthermore, when the imaging device 12 is made to perform the functions of the imaging support device 14 in this manner, at least one other CPU, at least one GPU, and / or at least one TPU may be used in place of or together with the CPU 62.

[0315] In the above embodiment, an example has been described in which the imaging support processing program 96 is stored in the storage 88, but the technology of the present disclosure is not limited to this. For example, the imaging support processing program 96 may be stored in a portable non-transitory storage medium such as an SSD or a USB memory. The imaging support processing program 96 stored in the non-transitory storage medium is installed in the computer 82 of the imaging support device 14. The CPU 86 executes imaging support processing in accordance with the imaging support processing program 96.

[0316] In addition, the imaging support processing program 96 may be stored in a storage device such as another computer or server device connected to the imaging support device 14 via the network 34, and the imaging support processing program 96 may be downloaded and installed on the computer 82 in response to a request from the imaging support device 14.

[0317] It is not necessary to store the entire image capture support processing program 96 in a storage device such as another computer or server device connected to the image capture support device 14, or in the storage 88; only a portion of the image capture support processing program 96 may be stored.

[0318] 2 includes a built-in controller 44, the technology of the present disclosure is not limited to this. For example, the controller 44 may be provided outside the imaging device 12.

[0319] In the above embodiment, the computer 82 is exemplified, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 82. Furthermore, instead of the computer 82, a combination of a hardware configuration and a software configuration may be used.

[0320] The hardware resources for executing the imaging support processing described in the above embodiments can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing imaging support processing by executing software, i.e., a program. Examples of processors include dedicated electrical circuits, such as FPGAs, PLDs, or ASICs, which are processors with circuit configurations designed specifically for executing specific processing. Each processor has built-in or connected memory, and each processor uses the memory to execute the imaging support processing.

[0321] The hardware resource that executes the imaging support process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the imaging support process may be a single processor.

[0322] As an example of a system configured with one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes the imaging support process. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute the imaging support process, on a single IC chip, as typified by SoCs. In this way, the imaging support process is realized using one or more of the various processors described above as hardware resources.

[0323] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The above-described imaging support process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0324] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0325] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0326] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. a processor; a memory connected to or embedded in the processor; The memory stores a first trained model; the first trained model is a trained model that outputs setting values ​​used for control related to imaging performed by an imaging device, The processor: performing a learning process using as training data a first image acquired by capturing an image using the imaging device and setting values ​​applied to control related to imaging performed by the imaging device when the first image was captured, thereby generating a second trained model that outputs setting values ​​used for the control related to imaging; performing identification processing based on a first setting value output from the first trained model when a second image is input to the first trained model and a second setting value output from the second trained model when the second image is input to the second trained model; Imaging support device.

2. The second image is stored in the memory. The imaging support device according to claim 1 .

3. The processor generates the second trained model by performing the learning process when a condition that the number of the first images has reached a first threshold is satisfied.

3. The imaging support device according to claim 1.

4. The training data is data including a plurality of images acquired by capturing images with the imaging device during a period from a specific time until the condition is satisfied, and a plurality of setting values ​​associated with the plurality of images and applied to the imaging device. The imaging support device according to claim 3 .

5. The processor performs the specific processing based on the first set value and the second set value when the condition is satisfied.

5. The imaging support device according to claim 3.

6. The processor performs the specifying process when the degree of difference between the first setting value and the second setting value is equal to or greater than a second threshold value. The imaging support device according to any one of claims 1 to 5.

7. The processor performs a predetermined process on the condition that the number of the first images reaches a third threshold. The imaging support device according to any one of claims 1 to 6.

8. The processor performs a predetermined process when the number of the first images obtained by performing the imaging in a first environment and used as the training data is equal to or greater than a fourth threshold, and the number of the first images obtained by performing the imaging in a second environment different from the first environment and used as the training data is equal to or less than a fifth threshold. The imaging support device according to any one of claims 1 to 6.

9. the imaging device is an interchangeable lens imaging device, The processor generates a plurality of the second trained models by performing the learning process for each type of interchangeable lens used to capture the first image. The imaging support device according to any one of claims 1 to 8.

10. When the interchangeable lens is attached to the imaging device, The processor performs processing using a second trained model generated by using, as the first image in the learning process, an image acquired by capturing an image using the imaging device to which the interchangeable lens is attached, among the plurality of second trained models. The imaging support device according to claim 9.

11. the imaging device includes a plurality of imaging systems; The processor generates a plurality of the second trained models by performing the learning process for each of the imaging systems used to capture the first image. The imaging support device according to any one of claims 1 to 10.

12. When an imaging system to be used for imaging is selected from the plurality of imaging systems, The processor performs processing using a second trained model generated by using, as the first image in the learning process, an image acquired by capturing an image with the imaging device using the imaging system selected from the plurality of second trained models. The imaging support device according to claim 11.

13. The processor: receiving an instruction to switch the plurality of imaging systems in a stepless manner; When the instruction is received, the use of the second trained model assigned to the imaging system before switching is continued in the imaging system after switching. The imaging support device according to claim 12.

14. The processor: a setting value for the learning process based on the scene when the first image was captured by the imaging device and information related to the selected imaging system; Based on the second setting value, the imaging device is caused to selectively use the plurality of imaging systems at a start timing of the imaging device.

14. The imaging support device according to claim 12 or 13.

15. The specific processing includes a first processing for reflecting the second setting value in the control. The imaging support device according to any one of claims 1 to 14.

16. The specific processing includes a second processing of storing the second trained model in a default storage device. The imaging support device according to any one of claims 1 to 15.

17. The identification process includes a third process of reflecting, in the control, an output of the first trained model or the second trained model, whichever is selected in accordance with an instruction received by the processor. The imaging support device according to any one of claims 1 to 16.

18. The identification process is a process including a fourth process of outputting first data for displaying, on a first display, a fourth image corresponding to an image obtained by inputting a third image to the first trained model and applying a first output result output from the first trained model to the third image, and a sixth image corresponding to an image obtained by inputting a fifth image to the second trained model and applying a second output result output from the second trained model to the fifth image. The imaging support device according to any one of claims 1 to 17.

19. The first data includes data for displaying the fourth image and the sixth image on the first display in a distinguishable manner. The imaging support device according to claim 18.

20. The first data includes data for displaying the fourth image on the first display in a state where the fourth image corresponds to first trained model identifying information that can identify the first trained model, and for displaying the sixth image on the first display in a state where the sixth image corresponds to second trained model identifying information that can identify the second trained model.

20. The imaging support device according to claim 18 or 19.

21. The fourth process includes, in accordance with an instruction received by the processor, a process of reflecting an output of the first trained model in the control when the fourth image is selected from the fourth image and the sixth image displayed on the first display, and a process of reflecting an output of the second trained model in the control when the sixth image is selected.

21. The imaging support device according to claim 18.

22. The identification process includes a fifth process of outputting second data for displaying, on a second display, time identification information that can identify the time when the second trained model was generated.

22. The imaging support device according to claim 1.

23. The second data includes data for displaying the period identification information on the second display in a state where the period identification information corresponds to a seventh image in which an output of the second trained model is reflected.

23. The imaging support device according to claim 22.

24. The identification process is a process including a sixth process of associating time identification information capable of identifying a time when the second trained model was generated with the second trained model.

24. The imaging support device according to claim 1.

25. The specific processing is processing including a seventh processing of reflecting an output of the second trained model in the control at a predetermined timing.

25. The imaging support device according to claim 1.

26. The predetermined timing is a timing when the imaging device is started, a timing when the number of captured images acquired by imaging with the imaging device becomes equal to or greater than a sixth threshold, a timing when charging of the imaging device is started, a timing when the operation mode of the imaging device transitions from a playback mode to a setting mode, or a timing when the captured images are rated in the playback mode.

26. The imaging support device according to claim 25.

27. the identification process is a process including an eighth process of correcting at least one of data input to the second trained model and output from the second trained model based on characteristics of the imaging device and characteristics of the other device when the second trained model is applied to another device that is an imaging device different from the imaging device.

27. The imaging support device according to claim 1.

28. The second trained model is accompanied by image sensor information including at least one of characteristic information indicating characteristics of each of the different image sensors involved in the second trained model and individual difference information indicating individual differences between the different image sensors; The processor identifies characteristics of the imaging device and characteristics of the other device using the image sensor information.

28. The imaging support device according to claim 27.

29. The identification process is a process including a ninth process of outputting third data for displaying, on a third display, a first processed image corresponding to an image obtained by inputting an eighth image to the second trained model and applying a third output result output from the second trained model to the eighth image, and an unprocessed image obtained without applying the third output result to the eighth image.

29. The imaging support device according to any one of claims 1 to 28.

30. The specific processing is processing including a tenth processing of outputting fourth data for displaying, on a fourth display, a brightness-adjusted image in which a fourth output result output from the second trained model is applied to the ninth image and the brightness is adjusted by inputting a ninth image to the second trained model, and an unprocessed image obtained without applying the fourth output result to the ninth image.

30. An imaging support device according to any one of claims 1 to 29.

31. A third processed image obtained by capturing an image while reflecting the output of the second trained model on the control is added with first associated information associated with the third processed image, The identification process is a process including an eleventh process of including information capable of identifying the second trained model in the first associated information.

31. The imaging support device according to claim 1.

32. A fourth processed image obtained by capturing an image while reflecting the output of the first trained model on the control is added with second associated information associated with the fourth processed image, The identification process includes a twelfth process of including information capable of identifying the first trained model in the second associated information.

32. The imaging support device according to any one of claims 1 to 31.

33. The setting value is at least one of a setting value related to white balance used in the imaging, a setting value related to exposure used in the imaging, a setting value related to focus used in the imaging, a setting value related to saturation used in the imaging, and a setting value related to gradation used in the imaging.

33. The imaging support device according to any one of claims 1 to 32.

34. a processor; a memory connected to or embedded in the processor; The memory stores a first trained model; the first trained model is a trained model that outputs setting values ​​used for control related to imaging performed by an imaging device, The processor: performing a learning process using as training data a first image acquired by capturing an image using the imaging device and setting values ​​applied to control related to imaging performed by the imaging device when the first image was captured, thereby generating a second trained model that outputs setting values ​​used for the control related to imaging; performing identification processing based on the degree of difference between the first trained model and the second trained model; Imaging support device.

35. a processor; a memory connected to or embedded in said processor; an imaging device body; The memory stores a first trained model; the first trained model is a trained model that outputs setting values ​​used for control related to imaging performed by the imaging device body, The processor: generating a second trained model that outputs setting values ​​used for control related to imaging by performing a learning process using as teacher data a first image acquired by imaging using the imaging device body and setting values ​​applied to control related to imaging performed by the imaging device body when the first image was acquired; performing identification processing based on a first setting value output from the first trained model when a second image is input to the first trained model and a second setting value output from the second trained model when the second image is input to the second trained model; Imaging device.

36. generating a second trained model that outputs setting values ​​used in control related to imaging performed by the imaging device by performing a learning process using, as teacher data, a first image acquired by imaging using an imaging device and setting values ​​applied to control related to imaging performed by the imaging device when the first image was acquired; and performing a specific process based on a first setting value output from a first trained model when a second image is input to the first trained model, the first setting value being a trained model that outputs a setting value used for control related to the imaging, and a second setting value output from the second trained model when the second image is input to the second trained model. Imaging support method.

37. On the computer, generating a second trained model that outputs setting values ​​used in control related to imaging performed by the imaging device by performing a learning process using, as teacher data, a first image acquired by imaging using an imaging device and setting values ​​applied to control related to imaging performed by the imaging device when the first image was acquired; and A program for executing a process including performing a specific process based on a first setting value output from a first trained model when a second image is input to the first trained model, the first setting value being a trained model that outputs setting values ​​used for control related to the imaging, and a second setting value output from the second trained model when the second image is input to the second trained model.

Images