Processing device, imaging device, processing method, and program

Abstract

A processing device according to the present invention comprises a processor. The processor executes image processing on a captured image that has been captured by an imaging device and, on the basis of the details of the image processing, generates control information for control of image capture.

Description

Processing apparatus, imaging apparatus, processing method, and program

[0001] The present disclosure relates to a processing apparatus, an imaging apparatus, a processing method, and a program.

[0002] Japanese Unexamined Patent Publication No. 2005-033504 discloses an image data generating apparatus that generates image data. The image data generating apparatus described in Japanese Unexamined Patent Publication No. 2005-033504 includes an imaging condition setting unit, an image data generating unit, an on-shooting image processing unit, an image data storage unit, an image data selecting unit, and an imaging condition changing unit. The imaging condition setting unit sets imaging conditions. The image data generating unit generates image data. The on-shooting image processing unit performs image processing on the image data generated using at least part of the set imaging conditions. The image data storage unit stores the image-processed image data in association with the imaging conditions. The image data selecting unit selects desired image data from the image data storage unit. The imaging condition display unit displays the imaging conditions associated with the selected image data. The imaging condition changing unit changes the displayed imaging conditions.

[0003] Japanese Unexamined Patent Publication No. 2024-046217 discloses an imaging system including an imaging apparatus and an image processing apparatus. The imaging apparatus described in Japanese Unexamined Patent Publication No. 2024-046217 includes a communication unit, an imaging unit, a control unit, and a setting unit. The communication unit performs data communication with an image processing apparatus connected via a communication network. The imaging unit captures an image of a subject to generate image data. The control unit transmits the image data to the image processing apparatus via the communication unit, and requests the image processing apparatus to perform image processing on the image data. The setting unit acquires setting information related to the image processing request in accordance with a user operation. Further, the control unit limits the image data to specific image data among the image data generated by the imaging unit based on the setting information acquired in accordance with the user operation, and requests the image processing apparatus to perform specific image processing.

[0004] On the other hand, the image processing apparatus described in Japanese Unexamined Patent Publication No. 2024-046217 is connected to the imaging apparatus via a communication network, and performs image processing requested by the imaging apparatus on image data received from the imaging apparatus.

[0005] One embodiment of the present disclosure provides a processing device, an imaging device, a processing method, and a program that enable imaging based on the content of image processing after image processing has been performed on an image captured by an imaging device.

[0006] A first aspect of the present disclosure is a processing device comprising a processor, the processor performing image processing on an image captured by an imaging device, and generating control information for controlling imaging based on the content of the image processing.

[0007] A second aspect of the present disclosure is an apparatus according to the first aspect, wherein the image processing includes a plurality of processes, and the content of the image processing includes the order and / or combination in which the plurality of processes are performed.

[0008] A third aspect of this disclosure is an processing apparatus according to the first or second aspect, wherein the image processing includes processing using a generating AI.

[0009] A fourth aspect of the present disclosure is a processing device relating to any one of the first to third aspects, wherein the control information includes support information that assists in imaging.

[0010] A fifth aspect of the present disclosure is a processing device relating to any one of the first to fourth aspects, wherein the control information includes information relating to imaging conditions that make image processing unnecessary.

[0011] A sixth aspect of the present disclosure is a processing device relating to any one of the first to fifth aspects, wherein control information is acquired by the imaging device through communication between the imaging device and the processing device.

[0012] A seventh aspect of the present disclosure is a processing device relating to any one of the first to sixth aspects, wherein the control information includes an captured image that is the target of image processing.

[0013] The eighth aspect of the present disclosure is an processing device relating to any one of the first to sixth aspects, wherein the image processing includes a plurality of processes, and the control information includes an captured image that is the target of the image processing and the order in which the plurality of processes are performed.

[0014] The ninth aspect of this disclosure is a processing device relating to any one of the first to eighth aspects, wherein the control information includes an captured image that is the target of image processing and a processed image obtained by performing image processing on the captured image.

[0015] The tenth aspect of the present disclosure is a processing apparatus according to any one of the first to ninth aspects, wherein the processor acquires information related to imaging and generates control information based on the content of image processing and the information related to imaging.

[0016] An eleventh aspect of this disclosure is a processing apparatus according to the tenth aspect, wherein the imaging information includes information regarding the imaging tendencies of the person who performed the imaging.

[0017] A twelfth aspect of the present disclosure is a processing device relating to any one of the first to eleventh aspects, wherein control information is applied to an imaging device different from the imaging device.

[0018] A thirteenth aspect of the present disclosure is a processing device according to any one of the first to twelfth aspects, wherein when an imaging device performs imaging based on settings applied to the imaging device without using at least a portion of the control information, the processor generates control information based on the content of the image processing and information about the settings.

[0019] A fourteenth aspect of the present disclosure is a processing device relating to any one of the first to thirteenth aspects, wherein the processor transmits control information to an external device of the processing device.

[0020] A 15th aspect of the present disclosure is a processing device relating to any one of the first to 14th aspects, wherein the control information is information used for updating an imaging device.

[0021] A sixteenth aspect of the present disclosure is a processing device relating to any one of the first to fifteenth aspects, wherein the image processing is performed outside the imaging device.

[0022] A 17th aspect of this disclosure is a processing apparatus according to the 16th aspect, wherein image processing is performed by external image processing software.

[0023] The eighteenth aspect of this disclosure is an processing device relating to any one of the first to fifteenth aspects, wherein the image processing is performed within an imaging device.

[0024] A 19th aspect of this disclosure is a processing apparatus according to the 18th aspect, wherein image processing is performed by image processing software within an imaging device.

[0025] A 20th aspect of the present disclosure is an imaging device that is updated using control information generated by a processing device according to any one of the first to 19th aspects.

[0026] A 21st aspect of the present disclosure is an imaging device according to a 20th aspect, having an imaging mode for performing imaging, wherein the imaging mode is controlled by control information.

[0027] A 22nd aspect of this disclosure is an imaging device according to the 20th or 21st aspect, which includes an engine for analyzing a subject or scene, and in which the results of the analysis by the engine are reflected in control information.

[0028] A 23rd aspect of the present disclosure is a processing method that includes performing image processing on an image captured by an imaging device, and generating control information to control imaging based on the content of the image processing.

[0029] A 24th aspect of this disclosure is a program for causing a computer to perform a process that includes performing image processing on an image captured by an imaging device, and generating control information to control imaging based on the content of the image processing.

[0030] This is a conceptual diagram showing an example of how an information processing system is used. This is a block diagram showing an example of the hardware configuration of an imaging device. This is a block diagram showing an example of the hardware configuration of an information processing device. This is a conceptual diagram showing an example of processing performed by the processor when the release key is in the imaging preparation instruction state. This is a conceptual diagram showing an example of processing performed by the processor when the release key is in the imaging start instruction state. This is a conceptual diagram showing an example of the processing content performed by the processor of an information processing device. This is a conceptual diagram showing an example of the processing content performed by the processor of an imaging device. This is a flowchart showing an example of the flow of imaging control processing. This is a flowchart showing an example of the flow of information generation processing. This is a conceptual diagram showing a first modified example of control information and an example of the processing content that generates the control information. This is a conceptual diagram showing a second modified example of control information and an example of the processing content that generates the control information. This is a conceptual diagram showing a third modified example of control information and an example of the processing content that generates the control information. This is a conceptual diagram showing a fourth modified example of control information and an example of the processing content that generates the control information. This is a conceptual diagram showing a fifth modified example of control information. This is a conceptual diagram showing a sixth modified example of control information. This is a conceptual diagram showing a seventh modified example of control information. This is a conceptual diagram showing an eighth modified example of control information. This is a conceptual diagram showing an example of the processing content in which the imaging mode is derived from the control information. This is a conceptual diagram illustrating an example of how image processing and control information generation are performed within an imaging device. It also illustrates a series of processes in which a processor within a computer issues processing instructions to an external device via a network, the external device executes the processing in accordance with the instructions, and the computer's processor receives the processing results from the external device.

[0031] Hereinafter, an example of an embodiment of the processing apparatus, imaging apparatus, processing method, and program related to this disclosure will be described with reference to the attached drawings.

[0032] First, let's explain the terminology used in the following explanation.

[0033] CPU stands for "Central Processing Unit". GPU stands for "Graphics Processing Unit". GPGPU stands for "General-Purpose computing on Graphics Processing Units". APU stands for "Accelerated Processing Unit". TPU stands for "Tensor Processing Unit". NPU stands for "Neural Processing Unit". DSP stands for "Digital Signal Processor". RAM stands for "Random Access Memory". DRAM stands for "Dynamic Random Access Memory". NVM stands for "Non-volatile memory". ROM stands for "Read Only Memory". EEPROM stands for "Electrically Erasable Programmable Read Only Memory". MRAM stands for "Magnetoresistive Random Access Memory". ReRAM stands for "Resistive Random Access Memory". FRAM (registered trademark) stands for "Ferroelectric Random Access Memory". ASIC stands for "Application Specific Integrated Circuit". PLD stands for "Programmable Logic Device". FPGA stands for "Field-Programmable Gate Array". CD-ROM stands for "Compact Disc Read Only Memory". DVD-ROM stands for "Digital Versatile Disc Read Only Memory". SSD stands for "Solid State Drive". USB stands for "Universal Serial Bus".EL stands for "Electro-Luminescence". UI stands for "User Interface". I / F stands for "Interface". AI stands for "Artificial Intelligence". LAN stands for "Local Area Network". WAN stands for "Wide Area Network". 5G stands for "5th Generation Mobile Communication System". Exif stands for "Exchangeable Image File Format". ISO stands for "International Organization for Standardization". JPEG stands for "Joint Photographic Experts Group". TIFF stands for "Tag Image File Format". CMOS stands for "Complementary Metal Oxide Semiconductor". CCD stands for "Charge Coupled Device". AF stands for "Auto Focus". AE stands for "Auto Exposure."

[0034] In the following description, a signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU, GPU, GPGPU, NPU, APU, TPU, or DSP.

[0035] In the following description, signed RAM refers to volatile memory that temporarily stores information and is used as work memory by the processor. An example of RAM is DRAM.

[0036] In the following description, signed NVM refers to non-volatile memory in which the stored information is retained even when the power is turned off, and is used for storing programs and data. Examples of NVM include flash memory, EEPROM, ROM, MRAM, ReRAM, or FRAM.

[0037] In the following description, a signed external interface (I / F) is responsible for the exchange of various types of information between multiple devices connected to each other. An example of an external interface is a USB interface. A communication interface, including a communication processor and an antenna, may also be applied to the external interface. The communication interface is responsible for communication between multiple computers. Examples of communication standards applied to the communication interface include wireless communication standards such as 5G, Wi-Fi®, or Bluetooth®.

[0038] In this specification, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0039] Figure 1 shows an example of the configuration of the information processing system 10. As an example, as shown in Figure 1, the information processing system 10 includes an imaging device 12 and an information processing device 14. The imaging device 12 and the information processing device 14 are used by a user 15. In this embodiment, the imaging device 12 is an example of the "imaging device" according to the disclosure. Also, in this embodiment, the information processing device 14 is an example of the "processing device" and "external part of the imaging device" according to the disclosure. Also, in this embodiment, the user 15 is an example of the "person who performed imaging" according to the disclosure.

[0040] The imaging device 12 is a consumer digital camera. Examples of consumer digital cameras include interchangeable-lens digital cameras and fixed-lens digital cameras. A consumer digital camera is merely an example; the invention also applies if the imaging device 12 is an industrial digital camera. The invention also applies if the imaging device 12 is an imaging device mounted on various electronic devices such as drive recorders, smart devices (e.g., smartphones), wearable devices, home appliances, cell observation devices, or modalities (e.g., endoscopes, ophthalmic observation devices, or surgical microscopes).

[0041] The imaging device 12 captures an image of the subject 16 when operated by the user 15. The imaging device 12 then generates an image file 18 by capturing the image of the subject 16. The image file 18 includes an image 20 showing the subject 16 and metadata 22, which is data related to the image 20 (in other words, data attached to the image 20). The file format of the image file 18 can be, for example, JPEG or TIFF. In this embodiment, the image 20 is an example of the "captured image" according to this disclosure.

[0042] An example of metadata 22 is data in Exif format. For example, metadata 22 includes various data used to generate the image 20, such as shutter speed, F-number, ISO sensitivity, focal length, exposure compensation amount (i.e., the amount of compensation used for exposure compensation), white balance information (e.g., color temperature and preset mode name), imaging mode, date and time of imaging, location of imaging, name of the manufacturer of the imaging device 12, model name of the imaging device 12, name of the lens used in the imaging device 12, resolution of the image 20, and date and time of creation of the image file 18. In this embodiment, metadata 22 is an example of "information related to imaging" according to this disclosure.

[0043] The image capturing device 12 is communicably connected to the information processing apparatus 14 via a network 24. Examples of the network 24 include WAN, LAN, and the like. The image capturing device 12 may be connected to the network 24 via a wireless method or a wired method, and the same applies to the information processing apparatus 14.

[0044] The image capturing device 12 transmits the image file 18 to the information processing apparatus 14 via the network 24. The information processing apparatus 14 receives the image file 18 transmitted from the image capturing device 12 via the network 24.

[0045] Here, an embodiment in which the image file 18 is transmitted and received via the network 24 is described, but this is merely an example. For example, the image file 18 may be transmitted and received via a portable storage medium such as a USB memory, or the image capturing device 12 may be directly connected to the information processing apparatus 14 so that the image file 18 is transferred from the image capturing device 12 to the information processing apparatus 14.

[0046] An example of the information processing apparatus 14 is a personal computer. Although a personal computer is exemplified here, this is merely an example, and the information processing apparatus 14 may be an information processing apparatus such as a server or a smart device.

[0047] A receiving device 26 such as a keyboard and a mouse, and a display 28 such as a liquid crystal display or an EL display are connected to the information processing apparatus 14. The receiving device 26 receives an instruction from a user 15. The display 28 displays, on a screen thereof, a processing result or the like obtained by the information processing apparatus 14.

[0048] The information processing apparatus 14 acquires an image 20 and metadata 22 from the image file 18. The information processing apparatus 14 displays the image 20 or the metadata 22 on the screen of the display 28 in accordance with the instruction received by the receiving device 26.

[0049] The information processing device 14 performs image processing 29 on the image 20 contained in the image file 18. For example, image processing 29 is achieved by launching conventionally known image processing software. An example of image processing 29 is processing to modify the image 20. Examples of processing to modify the image 20 include adjusting the brightness of the image 20, adjusting the saturation of the image 20, adjusting the white balance of the image 20, removing noise from the image 20 (so-called noise reduction), adjusting the resolution of the image 20, emphasizing or blurring the outlines of objects in the image 20, erasing or replacing the contents of the image 20, trimming the outline of the image 20, and adding visible information (an image, color, pattern, mark, character, or number different from the image 20) to the image 20. In the example shown in Figure 1, image processing 29 is performed on the image 20 in the image file 18, resulting in an example where the image 20 is made brighter overall and noise is removed from the image 20.

[0050] When image processing 29 is performed, metadata 22 may be displayed on the screen of the display 28 in accordance with the instructions received by the receiving device 26 and may be referenced by the user 15. In this case, the user 15 performs image processing 29 on the image 20 contained in the image file 18 while referring to the metadata 22.

[0051] Figure 2 shows an example of the hardware configuration of the imaging device 12 in a block diagram. As an example, as shown in Figure 2, the imaging device 12 includes a computer 30, an image sensor 32, a drive unit 34, a UI system 36, an external I / F 38, a photometering sensor 40, an optical system 42, and a mechanical shutter 44.

[0052] The computer 30 comprises a processor 46, an NVM 48, and RAM 50. The processor 46, NVM 48, and RAM 50 are connected to a bus 51. In this embodiment, the computer 30 is an example of the “engine” according to the disclosure.

[0053] The processor 46 executes the imaging control process. The NVM 48 stores the imaging control program 52. The processor 46 reads the imaging control program 52 from the NVM 48 and executes the read imaging control program 52 on the RAM 50. The imaging control process is realized by the execution of the imaging control program 52 by the processor 46.

[0054] The image sensor 32 is connected to the bus 51. An example of the image sensor 32 is a CMOS image sensor. Under the control of the processor 46, the image sensor 32 captures an image of the subject 16 (see Figure 1), generates RAW data 20A, and outputs it to the processor 46. Here, a CMOS image sensor is given as an example of the image sensor 32, but this is merely an example, and the image sensor 32 may be other types of image sensors such as a CCD image sensor.

[0055] The processor 46 performs various processes (for example, offset correction, white balance correction, demosaicing, color correction, gamma correction, color space conversion, luminance processing, color difference processing, and resizing) on ​​the RAW data 20A input from the image sensor 32 by executing imaging control processing. In this way, the processor 46 generates an image 20 (see Figure 1) and generates an image file 18 (see Figure 1) using the generated image 20.

[0056] The optical system 42 includes a lens 42A and an aperture 42B. The lens 42A captures subject light, which is light indicating the subject 16 (see Figure 1), and forms an image on the light-receiving surface of the image sensor 32. The lens 42A includes a movable lens that moves when powered. The movable lens includes a focusing lens (not shown), a magnification lens (not shown), and a blur-correcting lens (not shown). The focusing lens focuses on the subject 16 by moving in the optical axis direction. The magnification lens changes the focal length by moving in the optical axis direction. The blur-correcting lens corrects the misalignment of the optical axis by moving based on the detection result of vibrations applied to the imaging device 12 detected by a gyro sensor or the like (not shown). By correcting the misalignment of the optical axis, blur that appears in the image due to vibrations applied to the imaging device 12 is suppressed.

[0057] On the optical axis, the aperture 42B is positioned between the lens 42A and the light-receiving surface of the image sensor 32. The aperture 42B adjusts the amount of light. The aperture 42B opens and closes when power is received. By opening and closing the aperture 42B, the exposure and depth of field are controlled.

[0058] The mechanical shutter 44 is positioned between the aperture 42B and the light-receiving surface of the image sensor 32. The mechanical shutter 44 opens and closes when power is received. By opening and closing, the mechanical shutter 44 blocks or allows light from the optical system 42 to pass through. This controls the exposure.

[0059] The drive unit 34 is connected to the bus 51 and generates power under the control of the processor 46. The drive unit 34 applies the generated power to the lens 42A (for example, the movable lens described above) and the aperture 42B, thereby operating the lens 42A and the aperture 42B.

[0060] The UI device 36 is connected to the bus 51. The UI device 36 receives instructions from the user 15 (see Figure 1) and displays various information. The UI device 36 includes a reception device 36A and a display 36B.

[0061] The receiving device 36A receives instructions from the user 15 and outputs a signal indicating the received instruction to the processor 46. The processor 46 executes processing according to the signal input from the receiving device 36A.

[0062] The reception device 36A includes a plurality of hard keys (not shown). The reception device 36A also includes a plurality of soft keys (not shown). The soft keys are implemented by a touch panel and a display 36B.

[0063] The multiple hard keys and multiple soft keys include a so-called release key and various setting keys. The release key functions as an imaging preparation instruction unit and an imaging instruction unit, and can detect two-stage operations: an imaging preparation instruction state and an imaging start instruction state. For example, if the release key is a hard key, the imaging preparation instruction state refers to the state where it is pressed from the standby position to an intermediate position (half-press position), and the imaging start instruction state refers to the state where it is pressed beyond the intermediate position to the final pressed position (full-press position). Also, for example, if the release key is a soft key, the imaging preparation instruction state refers to the state where the user 15's finger is in contact with the release key, and the imaging start instruction state refers to the state where the user 15's finger has moved away from the state where it was in contact with the release key.

[0064] Here, an example is given in which the reception device 36A includes multiple hard keys and multiple soft keys, but this is merely one example, and the reception device 36A may also include a microphone for receiving voice instructions from the user 15, and / or a gesture detector for receiving gesture instructions from the user 15.

[0065] The display 36B is a liquid crystal display or an EL display, and under the control of the processor 46, it displays various information on the screen. Here, the display 36B is given as an example of a device that outputs various information, but this is merely an example, and the UI system device 36 may also include a speaker that outputs various information as sound, and / or a vibrator that generates vibrations indicating various information, in addition to the display 36B.

[0066] The photometering sensor 40 is connected to the bus 51. The photometering sensor 40 measures the brightness of the entire scene and outputs a photometric value indicating the brightness of the entire scene to the processor 46. The photometric value is used by the processor 46 for various processes, including automatic imaging (for example, imaging using AF and / or AE).

[0067] The external interface 38 is connected to the bus 51. The external interface 38 is connected to the memory card MC used in the imaging device 12 and is responsible for the exchange of various information between the memory card MC and the processor 46. That is, the processor 46 writes image files 18 etc. to the memory card MC via the external interface 38 and reads image files 18 etc. from the memory card MC via the external interface 38. The external interface 38 is also connected to the network 24 and is responsible for the exchange of various information between the information processing device 14 (see Figure 1) on the network 24 and the processor 46. That is, the processor 46 transmits image files 18 etc. to the information processing device 14 via the external interface 38 and acquires various information from the information processing device 14 via the external interface 38.

[0068] Figure 3 is a block diagram showing an example of the electrical hardware configuration of the information processing device 14. As shown in Figure 3, the information processing device 14 includes a computer 54, a UI system device 57, and an external I / F 58. The computer 54 includes a processor 60, an NVM 62, and RAM 64. The processor 60, NVM 62, and RAM 64 are connected to a bus 66. In this embodiment, the processor 60 is an example of a "processor" according to the disclosure, and the computer 54 is an example of a "computer" according to the disclosure.

[0069] The UI device 57 is connected to the bus 66. The UI device 57 receives instructions from the user of the information processing device 14 (for example, user 15 shown in Figure 1) and presents various information.

[0070] The UI system device 57 includes a reception device 26 and a display 28. The reception device 26 receives instructions from the user 15 and outputs a signal indicating the received instruction to the processor 60. The processor 60 operates according to the signal input from the reception device 26. The display 28 displays various information, including images, under the control of the processor 60.

[0071] The external interface 58 is connected to the bus 66. The external interface 58 is also connected to the network 24 and is responsible for the exchange of various information between the processor 60 and external devices on the network 24. In other words, the processor 60 transmits various information to external devices on the network 24 via the external interface 58, and obtains various information from external devices on the network 24 via the external interface 58.

[0072] Figure 3 shows an example of a configuration in which the external interface 58 of the information processing device 14 is connected to the external interface 38 of the imaging device 12, which is one of the external devices on the network 24, via the network 24. The image file 18 generated by the processor 46 of the imaging device 12 is transmitted from the external interface 38 to the information processing device 14 via the network 24 and received by the external interface 58 of the information processing device 14. The image file 18 received by the external interface 58 is then acquired by the processor 60 of the information processing device 14.

[0073] However, with conventionally known technologies, even if an external device of the imaging device 12 performs color correction or exposure adjustment using image processing software on the image 20 provided by the imaging device 12, the processing results from the image processing software are not fed back to subsequent imaging by the imaging device 12. As a result, if user 15 likes the processing results from the image processing software of the external device and wants to achieve those results at the imaging stage, user 15 needs to set imaging conditions and parameters used in development processing while considering the processing results from the image processing software of the external device. Furthermore, for beginners or novice users who are not familiar with handling the imaging device 12, making various settings for the imaging device 12 based on the processing results from the image processing software of the external device is an extremely difficult task.

[0074] Therefore, in light of these circumstances, in this embodiment, the information generation process is performed by the processor 60 of the information processing device 14. The image processing 29 shown in Figure 1 is one of the processes included in the information generation process. The NVM 62 stores the information generation program 68. In this embodiment, the information generation program 68 is an example of the "program" according to this disclosure. The processor 60 reads the information generation program 68 from the NVM 62 and executes the read information generation program 68 on the RAM 64. The information generation process is realized by the processor 60 executing the information generation program 68.

[0075] The NVM62 stores the generated AI70. The generated AI70 is an example of the "generated AI" related to this disclosure. The generated AI70 is used by the processor60. As will be described in detail later, the image processing 29 shown in Figure 1 includes processing using the generated AI70.

[0076] Figure 4 shows an example of part of the imaging control process (for example, the process executed by the processor 46 when the release key is in the imaging preparation instruction state). For example, when the user 15 gives an imaging preparation instruction to the imaging device 12, the processor 46 acquires various information used to derive the imaging conditions for automatic imaging. For example, as shown in Figure 4, the processor 46 acquires RAW data 20A generated by the image sensor 32. The processor 46 also acquires a photometric value 58 indicating the brightness of the entire scene from the photometric sensor 40. The processor 46 acquires a live view image 55 by generating a live view image 55 based on the RAW data 20A. The processor 46 displays the live view image 55 on the screen of the display 36B.

[0077] In the imaging device 12, information including the live view image 55 and photometric values ​​58 is used by the processor 46 to derive the imaging conditions for automatic imaging. That is, the processor 46 generates imaging conditions 56 based on the live view image 55 and photometric values ​​58. For example, the processor 46 identifies the brightness of the entire scene from the live view image 55 and photometric values ​​58, generates imaging conditions 56 corresponding to the identified brightness, and stores the generated imaging conditions 56 in the NVM 48.

[0078] For example, the generation of imaging conditions 56 can be achieved by using a rule-based approach that employs a table and / or calculation formulas that can derive imaging conditions 56 from the overall brightness of the scene. The use of a rule-based approach is merely one example; the generation of imaging conditions 56 may also be achieved by using a trained model obtained by optimizing a model (e.g., a neural network) through machine learning with various overall scene brightness values ​​and imaging conditions 56.

[0079] The imaging conditions 56 correspond to the imaging conditions used for exposure control by the imaging device 12 (for example, imaging conditions used in conventionally known AE). The imaging conditions 56 include settings related to the imaging device 12 when imaging is performed. The settings related to the imaging device 12 when imaging is performed are exposure conditions (the so-called three principles of exposure). In the example shown in Figure 4, an example of settings related to the imaging device 12 when imaging is performed is the F-number 56A applied to the aperture 42B (see Figure 2), the shutter speed 56B applied to the mechanical shutter 44 (see Figure 2), and the ISO sensitivity 56C applied to the image sensor 32. The live view image 55 and / or metering value 58 are also used for focus control by the imaging device 12 (for example, conventionally known AF).

[0080] The NVM 48 stores imaging conditions 72, multiple development system parameters 74, and multiple effect system parameters 76. Imaging conditions 72 are the imaging conditions currently set for the imaging device 12. Each time the processor 46 generates new imaging conditions 56, it updates the contents of imaging conditions 72 with the contents of the newly generated imaging conditions 56.

[0081] The multiple development system parameters 74 are various parameters used in the development process. Development process refers to the process of developing RAW data 20A (i.e., the process of generating an image file 18). The development process includes multiple processes. Examples of the multiple processes included in the development process include conventionally known processes such as offset correction, white balance, demosaicing, color correction, gamma correction, color space conversion, luminance filtering, color difference filtering, resizing, and compression.

[0082] The multiple development system parameters 74 are various parameters used in multiple processes included in the development process. An example of a parameter used in the offset correction process is the offset value such as the black level or pedestal level of the CMOS sensor. An example of a parameter used in the white balance process is the color temperature and / or color deviation. An example of a parameter used in the demosaicing process is the algorithm setting value when constructing an RGB image from RAW data 20A. An example of a parameter used in the color correction process is the color matrix. An example of a parameter used in the gamma correction process is the gamma value. An example of a parameter used in the color space conversion process is the parameter that manages the conversion to the color space. An example of a parameter used in the luminance filter process is the luminance filter parameter. An example of a parameter used in the color difference filter process is the color difference filter parameter. An example of a parameter used in the resizing process is the scaling factor and / or the number of pixels. An example of a parameter used in the compression process is the compression ratio and / or the parameter that manages the image quality level.

[0083] The multiple effect parameters 76 are various parameters used for multiple effect processing. Effect processing refers to the process of adding visual effects to the image 20. Adding visual effects to the image 20 refers to the process of adding effects to the image 20 that intentionally change the appearance of the image 20 by applying special effects and / or processing to the image 20, thereby deviating from the faithful representation of the original color reproduction and exposure reproduction.

[0084] The following are examples of multiple effect processing steps, from the first to the sixth. Note that steps 1 through 6 are merely examples, and other effect processing steps may also be used.

[0085] The first process is to remove noise from image 20 (so-called noise reduction). The second process is to adjust the resolution of image 20. The third process is to emphasize or blur the outlines of objects in image 20. The fourth process is to erase or replace the contents of image 20. The fifth process is to trim the outline of image 20. The sixth process is to add visible information (an image, color, pattern, mark, character, or number different from image 20) to image 20.

[0086] Examples of parameters used in the first process include noise reduction intensity, threshold, and / or spatial filter radius. Examples of parameters used in the second process include resolution, scaling factor, and / or number of pixels. Examples of parameters used in the third process include sharpness intensity, blur radius, and / or edge threshold. Examples of parameters used in the fourth process include parameters that control the range to erase the contents of image 20 and / or the range to replace the contents of image 20. Examples of parameters used in the fifth process include cropping range and / or aspect ratio. Examples of parameters used in the sixth process include watermark image identifier, transparency, positional coordinates, and / or font size. Although multiple effect processes are listed here, this is merely an example, and this disclosure is valid even with only one effect process. Furthermore, this disclosure is valid even without any effect processes.

[0087] Figure 5 shows an example of part of the imaging control process (for example, the process executed by the processor 46 when the release key is in the imaging start instruction state). As shown in Figure 5, the processor 46 performs imaging using the imaging conditions 72. As imaging is performed, RAW data 20A is generated by the image sensor 32, so the processor 46 acquires the RAW data 20A from the image sensor 32.

[0088] The processor 46 generates an image file 18 by performing a development process on the RAW data 20A using multiple development system parameters 74.

[0089] The processor 46 performs multiple effect processing on the image 20 contained in the image file 18 generated by the development process, using multiple effect parameters 76.

[0090] After performing multiple effect processing steps, the processor 46 outputs the image file 18 to a default output destination. A first example of a default output destination is a memory card MC. A second example of a default output destination is an information processing device 14.

[0091] Figure 6 shows an example of information generation processing. As shown in Figure 6, in the information processing device 14, the processor 60 acquires an image file 18 from the imaging device 12. The processor 60 acquires the image 20 contained in the image file 18 as the image 20B to be processed by image processing 29.

[0092] The processor 60 performs image processing 29 on the image 20B. Image processing 29 is implemented by image processing software (for example, conventionally known image editing software).

[0093] Image processing 29 includes multiple processes. In the example shown in Figure 6, as an example of the multiple processes included in image processing 29, several processes are illustrated, including brightness adjustment process 29A, saturation adjustment process 29B, white balance process 29C, and effect process 29D.

[0094] Brightness adjustment process 29A adjusts the overall brightness of image 20B. Saturation adjustment process 29B adjusts the saturation of image 20B after brightness adjustment process 29A has been performed. White balance process 29C adjusts the color tone of image 20B after saturation adjustment process 29B has been performed.

[0095] Effect processing 29D is a process that adds visual effects to the image 20B after white balance processing 29C and the like have been performed. Effect processing 29D is a process that uses generation AI 70. Generation AI 70 is a trained model obtained by optimizing the model (e.g., a neural network) through machine learning using a dataset (so-called training data) that includes example images and ground truth images obtained by adding visual effects to the example images (i.e., performing effect processing).

[0096] A first example of effect processing 29D is a process of the same type as at least one of the first to sixth processes described above, and which achieves a higher level of accuracy than that performed within the imaging device 12. A second example of effect processing 29D is a different type of effect processing than the first to sixth processes described above. Examples of different types of effect processing than the first to sixth processes described above include vignette processing, contrast adjustment processing, cropping processing, color replacement processing, high dynamic range processing, and mosaic processing.

[0097] Image processing 29 is performed on image 20B to obtain the processed image 20C. The processor 60 generates an image file containing image 20C. The image file is an image file in a default format (for example, a JPEG file or a TIFF file).

[0098] The processor 60 executes the image file output process 80. The image file output process 80 is the process of outputting an image file containing image 20C. An example of an output destination for the image file 18 is the imaging device 12. The image file 18 may also be output to the NVM 62 and / or a server on the network 24.

[0099] Meanwhile, the processor 60 generates control information 84A to control imaging by the imaging device 12 based on the content of the image processing 29. For example, the control information 84A is information that can control the imaging conditions 72, development system parameters 74, and / or effect system parameters 76 shown in Figures 4 and 5. In this case, the control of imaging by the imaging device 12 is achieved by controlling the imaging conditions 72, development system parameters 74, and / or effect system parameters 76 based on the control information 84A.

[0100] The image processing 29 includes multiple processing parameters 82. These multiple processing parameters 82 are various parameters used in multiple processes included in the image processing 29, such as parameters used in brightness adjustment processing 29A, parameters used in saturation adjustment processing 29B, parameters used in white balance processing 29C, and parameters used in effect processing 29D.

[0101] Furthermore, the content of the image processing 29 includes the sequence in which the brightness adjustment process 29A, saturation adjustment process 29B, white balance process 29C, and effect process 29D are executed, in addition to the multiple processing parameters 82. The content of the image processing 29 also includes combinations of the multiple processes included in the image processing 29.

[0102] In this embodiment, the control information 84A includes a plurality of processing parameters 82, information that can identify the order in which the brightness adjustment process 29A, saturation adjustment process 29B, white balance process 29C, and effect process 29D, etc., were executed, and information that can identify the combination of a plurality of processes included in the image processing 29. This is merely an example, and the control information 84A may be encrypted information containing the plurality of processing parameters 82, the order in which the brightness adjustment process 29A, saturation adjustment process 29B, white balance process 29C, and effect process 29D, etc., were executed, and the information that can identify the combination of a plurality of processes included in the image processing 29.

[0103] Here, an example is given in which the control information 84A is information that includes multiple processing parameters 82. However, this is merely one example, and the control information 84A may include information based on multiple processing parameters 82, as long as it is information generated based on multiple processing parameters 82 and is capable of controlling the imaging of the imaging device 12. A first example of information based on multiple processing parameters 82 is information in which the multiple processing parameters 82 have been converted into imaging conditions 72, development system parameters 74, and / or effect system parameters 76. A second example of information based on multiple processing parameters 82 is information in which the multiple processing parameters 82 have been encrypted.

[0104] In this embodiment, an example is given in which the content of the image processing 29 includes both the order in which the brightness adjustment process 29A, saturation adjustment process 29B, white balance process 29C, and effect process 29D are executed, and a combination of multiple processes included in the image processing 29. However, this is merely an example, and the content of the image processing 29 may include either the order in which the brightness adjustment process 29A, saturation adjustment process 29B, white balance process 29C, and effect process 29D are executed, or a combination of multiple processes included in the image processing 29.

[0105] Furthermore, the content of the image processing 29 may not include the order in which the brightness adjustment process 29A, saturation adjustment process 29B, white balance process 29C, and effect process 29D are executed, nor the combination of multiple processes included in the image processing 29, but may include multiple processing parameters 82 and / or information based on the multiple processing parameters 82.

[0106] When the processor 60 generates control information 84A, it executes control information output processing 86. Control information output processing 86 is the process of outputting the control information 84A. The output destination of the control information 84A is the imaging device 12. The control information 84A may also be output to the NVM 62 and / or to a server on the network 24.

[0107] Figure 7 shows an example of part of the imaging control process. As shown in Figure 7, the NVM 48 of the imaging device 12 stores an information conversion table 88. The information conversion table 88 is a table that defines the correspondence between control information 84A and imaging device parameters. Here, imaging device parameters refer to imaging conditions 72, multiple development system parameters 74, and multiple effect system parameters 76. In the following, for the sake of convenience, when it is not necessary to distinguish between imaging conditions 72, multiple development system parameters 74, and multiple effect system parameters 76, they will be referred to as "imaging device parameters".

[0108] The imaging device parameters are parameters that control the imaging device 12 so as to achieve, within the range of the imaging and image processing capabilities of the imaging device 12, an image 20C (see Figure 6) obtained by executing image processing 29 by the processor 60, assuming that the control information 84A associated with the imaging device parameters is generated by the processor 60 of the information processing device 14, with the same quality as, or as close as possible to, the quality of image 20C.

[0109] In the imaging device 12, the processor 46 acquires control information 84A from the information processing device 14. The processor 46 refers to the information conversion table 88 and converts the control information 84A acquired from the information processing device 14 into imaging device parameters. That is, the processor 46 derives imaging device parameters corresponding to the control information 84A acquired from the information processing device 14 from the information conversion table 88.

[0110] The processor 46 then updates the imaging device 12 using the imaging device parameters derived from the information conversion table 88. This update is achieved by updating the contents of the imaging device parameters stored in the NVM 48 with the contents of the imaging device parameters derived from the information conversion table 88.

[0111] Next, the operation of the information processing system 10 will be explained with reference to Figures 8 and 9.

[0112] First, an example of the flow of imaging control processing performed by the processor 46 of the imaging device 12 will be explained with reference to Figure 8.

[0113] In the imaging control process shown in Figure 8, first, in step ST10, the processor 46 determines whether the release key is in the imaging preparation instruction state. If the release key is not in the imaging preparation instruction state in step ST10, the determination is denied, and the imaging control process proceeds to step ST20. If the release key is in the imaging preparation instruction state in step ST10, the determination is affirmed, and the imaging control process proceeds to step ST12.

[0114] In step ST12, the processor 46 acquires the live view image 55 and the photometric value 58 (see Figure 4). After the processing in step ST12 is completed, the imaging control process moves on to step ST14.

[0115] In step ST14, the processor 46 generates imaging conditions 56 based on the live view image 55 and photometric values ​​58 acquired in step ST12. The processor 46 then updates the imaging conditions 72 stored in the NVM 48 by replacing the contents of the imaging conditions 72 with the contents of the imaging conditions 56 (see Figure 4). After the processing in step ST14 is completed, the imaging control process moves on to step ST16.

[0116] In step ST16, the processor 46 determines whether or not an imaging start command has been received by the release key. If, in step ST16, an imaging start command has not been received by the release key, the determination is denied and the imaging control process proceeds to step ST20. If, in step ST16, an imaging start command has been received by the release key, the determination is affirmed and the imaging control process proceeds to step ST18.

[0117] In step ST18, the processor 46 generates an image file 18 by sequentially executing imaging using imaging conditions 72, development processing using multiple development system parameters 74, and multiple effect processing using multiple effect system parameters 76. The processor 46 then outputs the generated image file 18 to the information processing device 14. After the processing in step ST18 is completed, the imaging control process moves to step ST20.

[0118] When the image file 18 is output to the information processing device 14 by the execution of step ST18, the information generation process shown in Figure 9 is executed by the processor 60 of the information processing device 14. When step ST62, which is included in the information generation process shown in Figure 9, is executed by the processor 60 of the information processing device 14, control information 84A is output from the information processing device 14 to the imaging device 12.

[0119] Therefore, in step ST20, the processor 46 determines whether or not the control information 84A output from the information processing device 14 to the imaging device 12 has been received by the external I / F 38 (see Figure 2). If, in step ST20, the control information 84A output from the information processing device 14 to the imaging device 12 has not been received by the external I / F 38, the determination is denied and the imaging control process proceeds to step ST24. If, in step ST20, the control information 84A output from the information processing device 14 to the imaging device 12 has been received by the external I / F 38, the determination is affirmed and the imaging control process proceeds to step ST22.

[0120] In step ST22, the processor 46 performs conversion processing and update processing (see Figure 7). Conversion processing refers to the process in which the processor 46 refers to the information conversion table 88 and converts the control information 84A received by the external I / F 38 in step ST20 into imaging device parameters. Update processing refers to the process in which the processor 46 updates the imaging device 12 using the imaging device parameters obtained by the conversion processing (in other words, the process of updating the contents of the imaging device parameters stored in the NVM 48 with the contents of the imaging device parameters obtained by the conversion processing). After the processing in step ST22 is executed, the imaging control processing moves to step ST24.

[0121] In step ST24, the processor 46 determines whether the conditions for terminating the imaging control process have been met. One example of a condition for terminating the imaging control process is that an instruction to terminate the imaging control process has been received by the receiving device 36A. If the conditions for terminating the imaging control process are not met in step ST24, the determination is denied, and the imaging control process proceeds to step ST10. If the conditions for terminating the imaging control process are met in step ST24, the determination is affirmed, and the imaging control process terminates.

[0122] Next, an example of the flow of information generation processing performed by the processor 60 of the information processing device 14 will be described with reference to Figure 9. The flow of information generation processing shown in Figure 9 is an example of the "processing method" related to this disclosure.

[0123] In the information generation process shown in Figure 9, first, in step ST50, the processor 60 determines whether the image file 18 output from the imaging device 12 to the information processing device 14, which was generated when the processing in step ST18, included in the imaging control process shown in Figure 7, was executed by the processor 46 of the imaging device 12, was received by the external I / F 58 (see Figure 3). If, in step ST50, the image file 18 output from the imaging device 12 to the information processing device 14 was not received by the external I / F 58, the determination is denied, and the information generation process proceeds to step ST64. If, in step ST50, the image file 18 output from the imaging device 12 to the information processing device 14 was received by the external I / F 58, the determination is affirmed, and the information generation process proceeds to step ST52.

[0124] In step ST52, the processor 60 acquires image 20B from image file 18 received by the external I / F 58 in step ST50 (see Figure 6). After the processing in step ST52 is completed, the information generation process moves on to step ST54.

[0125] In step ST54, the processor 60 performs image processing 29 on the image 20B acquired in step ST52 (see Figure 6). This results in image 20C (see Figure 6). After the processing in step ST54 is completed, the information generation process moves on to step ST56.

[0126] In step ST56, the processor 60 executes the image file output process 80. As a result, the image 20C is output to the default output destination. After the process in step ST56 is executed, the information generation process moves on to step ST58.

[0127] In step ST58, the processor 60 acquires multiple processing parameters 82 (see Figure 6). After the processing in step ST58 is completed, the information generation process proceeds to step ST60.

[0128] In step ST60, the processor 60 generates control information 84A based on the multiple processing parameters 82 acquired in step ST58 (see Figure 6). After the processing in step ST60 is completed, the information generation process moves on to step ST62.

[0129] In step ST62, the processor 60 executes the control information output process 86. As a result, the control information 84A generated in step ST60 is output to the imaging device 12. After the processing in step ST62 is completed, the information generation process moves on to step ST64.

[0130] In step ST64, the processor 60 determines whether the conditions for terminating the information generation process have been met. One example of a condition for terminating the information generation process is that an instruction to terminate the information generation process has been received by the receiving device 26. If the conditions for terminating the information generation process are not met in step ST64, the determination is denied, and the information generation process proceeds to step ST50. If the conditions for terminating the information generation process are met in step ST64, the determination is affirmed, and the information generation process terminates.

[0131] As explained above, the information processing device 14 performs image processing 29 on the image 20B obtained by the imaging device 12. Then, control information 84A is generated based on the content of the image processing 29. The control information 84A is information that controls the imaging of the imaging device 12. Therefore, after the image processing 29 is performed on the image 20B obtained by the imaging device 12, imaging based on the content of the image processing 29 can be realized.

[0132] As a result, the content of the image processing 29 can be reflected in subsequent imaging, thus increasing the consistency between imaging and image processing 29. Furthermore, conventionally, the user 15 had to manually change the settings of the imaging device 12 to reflect the adjustments made in the image processing software back into the imaging settings. However, in this embodiment, control information 84A based on the content of the image processing 29 for image 20B is generated and imaging is controlled, significantly reducing the effort required from the user 15. Additionally, based on the content of the image processing 29, imaging device parameters close to the desired final result can be pre-set on the imaging device 12 side, reducing the amount of editing work required in subsequent stages and making it easier to obtain an image that is nearly complete from the preview stage at the imaging site. Moreover, while conventional automatic imaging is based on generalized logic, in this embodiment, the content of the individual image processing 29 actually performed by the user 15 is reflected in the imaging device parameters used for subsequent imaging, making it easier to obtain an image 20 that matches the user 15's preferences and imaging tendencies. Furthermore, while studio imaging or mass production imaging (e.g., product imaging) requires repeating the same editing process many times, in this embodiment, the settings already obtained in image processing 29 can be reflected on the imaging side, thus streamlining the workflow for mass imaging.

[0133] Image processing 29 includes multiple processes, and the final result may change depending on the order in which these processes are executed. Similarly, different combinations of processes will also result in different final outcomes. Therefore, in this embodiment, by reflecting the execution order and combination of the multiple processes included in image processing 29 in the control information 84A, the final effect intended by the person who designed the multiple processes and combinations included in image processing 29 (e.g., user 15) can be faithfully incorporated into subsequent imaging. As a result, the effects of each individual process included in image processing 29 can be accurately reproduced, achieving higher fidelity and imaging efficiency. Furthermore, since the order and combination of multiple processes, such as exposure, color correction, noise reduction, and cropping, are applied to subsequent imaging, editing techniques and special filter application orders devised by user 15 can be incorporated into subsequent imaging.

[0134] Furthermore, the information processing device 14 includes effect processing 29D in its image processing 29, and effect processing 29D is implemented by the generation AI 70. Therefore, the complex editing intentions of the generation AI 70 can be applied to subsequent image captures. In addition, the taste preferred by the generation AI 70 (for example, the degree to which the background is blurred in portraits) can be applied as the initial settings of the imaging device 12. Moreover, the new image expression by the generation AI 70 can be reflected in the imaging device parameters in real time.

[0135] Furthermore, in this embodiment, control information 84A is acquired by the imaging device 12 through communication between the imaging device 12 and the information processing device 14. Therefore, the contents of the image processing 29 are immediately reflected in the imaging device 12 without waiting for user 15's operation. In addition, even when away from home, user 15 can update imaging device parameters in real time by having the image processing software of the information processing device 14 and the imaging device 12 work together. Also, if user 15 decides to "increase the exposure a little more and take the picture again" while image processing 29 is being performed on the information processing device 14, the new settings based on the processing results of image processing 29 are immediately sent to the imaging device 12, and re-imaging can be performed.

[0136] Furthermore, in this embodiment, control information 84A is used to update the imaging device 12. Therefore, if a specific process is performed repeatedly, the imaging device 12 can be updated to enhance the functions related to that specific process. For example, if backlight compensation is used frequently, a more powerful high dynamic range imaging mode can be added via firmware. In addition, the imaging device 12 can be evolved through the content of image processing 29. That is, the imaging device 12 can improve its performance over the long term by continuously learning the user's 15 imaging tendencies and the latest image processing 29 techniques. Also, because updates are possible, even if new image processing techniques are developed in the information processing device 14, there is room for them to be incorporated into the imaging device 12. As a result, the performance of the imaging device 12 can be maintained and improved over the long term. Furthermore, by appropriately updating the functions of the imaging device 12, which were previously fixed, based on analysis results from external software, etc., the user 15 can continue to use the imaging device 12 that incorporates new imaging technologies and / or image processing techniques. As a result, users of the imaging device 12 can always use advanced imaging settings without having to frequently replace the device. Furthermore, if the updated information can be used for different imaging devices 12, the user's preferences and imaging know-how can be shared among multiple imaging devices 12.

[0137] Furthermore, in this embodiment, image processing 29 is performed on an information processing device 14, which is external to the imaging device 12. Therefore, image processing 29 can be performed utilizing the powerful computing capabilities of the information processing device 14, and the processing results of image processing 29 can be fed back to the imaging device 12. As a result, it is possible to contribute to power saving and ensuring real-time performance on the imaging device 12 side. In addition, since it is external processing, a high-performance image processing engine (e.g., generation AI) can be upgraded at any time. As a result, it is possible to benefit from advancements in external processing without having to replace the imaging device 12.

[0138] Furthermore, in this embodiment, image processing 29 is performed by image processing software on an information processing device 14, which is external to the imaging device 12. Therefore, the imaging device 12 can be linked with commonly used software. In addition, the user 15 can link the settings of the imaging device 12 with tools they are familiar with. External software can be enhanced with plugins and other means, and advanced processing results can be reflected in the imaging device parameters of the imaging device 12.

[0139] In the above embodiment, an example was given in which imaging device parameters are derived using the information conversion table 88. However, this is merely one example, and a generating AI that has learned combinations of control information 84A and imaging device parameters may generate imaging device parameters corresponding to the control information 84A.

[0140] Furthermore, while the above embodiment provides an example where the control information 84A includes a plurality of processing parameters 82, this is merely one example. The control information 84A may also include information regarding the three principles of exposure (i.e., the F-number 56A, shutter speed 56B, and ISO sensitivity 56C of the imaging device 12) derived from the plurality of processing parameters 82, the order of the plurality of processes included in the image processing 29, and / or combinations of the plurality of processes included in the image processing 29 using a rule-based or AI-based method. This allows for direct changes to the settings during imaging, such as "increase brightness in image processing 29" ⇒ "slow down the shutter speed or increase the ISO sensitivity during the next imaging session." As a result, it becomes easier to obtain optimal exposure without image processing software. Also, for example, the imaging device 12 can automatically adjust for the trade-off where increasing the ISO sensitivity too much increases noise. Therefore, the user 15 can achieve imaging using optimal imaging conditions 72 in conjunction with image processing 29 without requiring complex knowledge.

[0141] Furthermore, in the above embodiment, when control information 84A is generated by the information processing device 14, it is forcibly transmitted to the imaging device 12, and imaging device parameters corresponding to the control information 84A are generated. However, this is merely one example. For example, the imaging device 12 may be configured to selectively set between an allow mode that permits imaging control based on the control information 84A acquired from the information processing device 14, and a reject mode that rejects imaging control based on the control information 84A acquired from the information processing device 14.

[0142] This allows the imaging device 12 to progressively review the control information 84A transmitted from the information processing device 14, enabling flexible selection such as "I only want to adopt color correction and exposure" or "I don't need to change the white balance." Furthermore, since no two situations are exactly alike, the feedback from the control information 84A may not always be effective. Therefore, users can quickly switch between "acceptable mode" for automatic application and "rejection mode" to maintain the previous manual settings, depending on the situation. As a result, the user 15 can operate the imaging device 12 in both fully automatic and partially automatic modes, significantly improving usability. Additionally, the "rejection mode" prevents the risk of unexpected results from automatic settings based on the control information 84A. For example, if the user 15 needs to quickly fix a specific setting, they can temporarily switch to reject mode to block unauthorized changes. Moreover, the availability of fully automatic, partially approved, and completely rejected options allows users from beginners to advanced levels to operate the system in a style that suits them.

[0143] Furthermore, while the above embodiment shows an example where control information 84A generated by the processor 60 of the information processing device 14 is output from the information processing device 14 to the imaging device 12, this is merely one example. For example, the processor 60 may upload the control information 84A to a cloud server or the like. In this way, the control information 84A can be obtained from any device that is authorized to use the control information 84A. In addition, it becomes easier to share and synchronize settings among multiple imaging devices 12 or among multiple users 15. Furthermore, it becomes possible to analyze the transmitted control information 84A on the cloud server or the like and send a new algorithm back to the imaging device 12, thereby realizing higher performance feedback. In addition, users 15 can always use the latest and most optimal control information 84A.

[0144] Furthermore, although the above embodiment describes an example in which control information 84A is generated based on a plurality of processing parameters 82, the disclosure is not limited thereto, and control information may be generated based on a plurality of processing parameters 82 and information related to imaging. In this case, for example, as shown in Figure 10, the processor 60 acquires metadata 22 contained in the image file 18 and generates control information 84B based on the plurality of processing parameters 82 and the metadata 22. The control information 84B thus generated is handled in the same manner as the control information 84A described in the above embodiment.

[0145] For example, metadata 22 includes information about imaging performed by the imaging device 12, such as information indicating that continuous shooting was performed, or information indicating that there are images deleted immediately after imaging. Therefore, control information 84B is generated based on multiple processing parameters 82 and metadata 22. For example, if continuous shooting is performed, or if there are images deleted immediately after imaging, information indicating that continuous shooting was performed or information indicating that there are images deleted immediately after imaging is reflected in the control information 84B.

[0146] This allows the imaging device 12 to determine what settings are necessary and unnecessary when continuous shooting is performed or when images are deleted immediately after imaging. Furthermore, for example, if the metadata 22 includes subject information or the number of shutter operations, the subject information or the number of shutter operations is reflected in the control information 84B, enabling imaging using more optimal imaging conditions 72. As a result, the accuracy of feedback from the information processing device 14 to the imaging device 12 can be improved from both the user's operation history and the content of the image processing 29. In the example shown in Figure 10, metadata 22 is an example of "information related to imaging" as per the Disclosure, and control information 84B is an example of "control information" as per the Disclosure.

[0147] The metadata 22 contains various types of information, as shown in Figures 11 to 13.

[0148] In the example shown in Figure 11, the metadata 22 includes imaging trend information 22A. The imaging trend information 22A is information about the user 15's imaging trends. In this case, the processor 60 generates control information 84C based on the metadata 22 including the imaging trend information 22A and a plurality of processing parameters 82. The control information 84C thus generated is handled in the same way as the control information 84A described in the above embodiment.

[0149] Examples of imaging tendency information 22A include information indicating "a user who always takes slightly underexposed images" or "a user who has difficulty keeping the image level and tends to tilt the image." Therefore, control information 84C is generated based on metadata 22 including imaging tendency information 22A and a plurality of processing parameters 82, so that information indicating "a user who always takes slightly underexposed images" or "a user who has difficulty keeping the image level and tends to tilt the image" is reflected in the control information 84C.

[0150] As a result, the imaging device 12 can learn information about the user 15's imaging tendencies, such as "a user who always takes slightly underexposed images" or "a user who has difficulty with horizontal lines and tends to tilt images." Consequently, the imaging device 12 can predict the user 15's imaging tendencies (in other words, the user 15's imaging habits) and automatically correct them during subsequent imaging, resulting in smoother imaging. Furthermore, by understanding the operating modes and settings that the user 15 frequently uses, the imaging device 12 can achieve more accurate switching of operating modes. Moreover, the imaging device 12 can absorb and optimize the preferences of each individual user 15. In the example shown in Figure 11, the imaging tendency information 22A is an example of "information about the imaging tendencies of the person who performed the imaging" as per this disclosure, and the control information 84C is an example of "control information" as per this disclosure.

[0151] In the example shown in Figure 12, the metadata 22 includes setting information 22B. The setting information 22B is information about the settings when the imaging device 12 performs imaging based on the settings applied to the imaging device 12 without using at least a part of the control information 84A (i.e., information about the settings applied to the imaging device 12). In this case, the processor 60 generates control information 84D based on the metadata 22 including the setting information 22B and a plurality of processing parameters 82. The control information 84D thus generated is handled in the same way as the control information 84A described in the above embodiment.

[0152] As shown in Figure 12, control information 84D is generated based on metadata 22 including setting information 22B and a plurality of processing parameters 82. For example, if control information 84A recommends "exposure +0.5", but user 15 takes an image with "exposure +1.0", the information processing device 14 can learn this fact. As a result, the information processing device 14 can generate control information 84D that is closer to "exposure +1.0" from the next time onward, and can continuously reflect user 15's preferences. In addition, depending on the scene or subject, user 15 may choose not to use control information 84A, so by incorporating the circumstances under which control information 84A was not used and updating control information 84D from control information 84A to control information 84D, the information processing device can always keep up with user 15's latest intentions. In the example shown in Figure 12, setting information 22B is an example of "setting information" related to this disclosure, and control information 84D is an example of "control information" related to this disclosure.

[0153] In the example shown in Figure 13, the metadata 22 includes analysis result information 22C. The analysis result information 22C is information indicating the results obtained by the processor 46 of the imaging device 12 analyzing the subject or scene by performing conventionally known image analysis processing (for example, image analysis processing by template matching or image analysis processing using AI). In this case, the processor 60 generates control information 84E based on the metadata 22 including the analysis result information 22C and a plurality of processing parameters 82. The control information 84E thus generated is handled in the same way as the control information 84A described in the above embodiment.

[0154] As shown in Figure 13, control information 84E is generated based on metadata 22 containing analysis result information 22C and a plurality of processing parameters 82. Based on the control information 84E, the imaging device parameters are determined and set. For example, if user 15 frequently applies an exposure compensation of +0.5 to photos of flowers, the imaging device 12 can automatically apply the optimal setting for each scene or subject, such as setting the exposure to +0.5 only when it recognizes "flowers" in the image analysis process. Furthermore, for example, no compensation is applied when subjects other than flowers are captured, thus achieving both appropriate scene discrimination and appropriate compensation. In other words, unnecessary compensation, such as overexposure caused by applying the same +0.5 exposure compensation to subjects other than flowers, is avoided. In addition, the editing patterns established by user 15 in the image processing software of the information processing device 14 (editing patterns such as exposure +0.5 for flowers and white balance correction for people) can be reflected only when detected by the processor 46 of the imaging device 12, thus eliminating the need for user 15 to switch settings each time. Furthermore, unlike conventional general-purpose automatic imaging, the user's (15) actual image processing know-how (e.g., preferences for color and / or brightness) can be directly applied to the subject or scene. As a result, more detailed customization and reproducibility can be expected.

[0155] Although control information 84A to 84E has been given as an example above, this disclosure is not limited thereto, and may also be control information 84F to 84I, as shown in Figures 14 to 17. Control information 84F to 84I is an example of "control information" related to this disclosure.

[0156] In the example shown in Figure 14, control information 84F is generated by the processor 60. The control information 84F includes image 20B. The control information 84F configured in this way is handled in the same way as the control information 84A described in the above embodiment.

[0157] Because the control information 84F includes image 20B, the imaging device 12 can refer to image 20B which is the target of image processing 29, thereby enabling accurate analysis of the image 20B's characteristics (e.g., brightness or subject). This also allows for the derivation of highly accurate imaging device parameters. Furthermore, by allowing imaging devices 12 of different models to refer to the same image 20B, conversion and adjustments to achieve similar results become easier even with imaging devices 12 of different models. Moreover, it becomes easier to share settings based on the content of image 20B, regardless of manufacturer or model. Note that image 20B is an example of an "imaging image targeted for image processing" according to this disclosure.

[0158] In the example shown in Figure 15, control information 84G is generated by the processor 60. The control information 84G includes images 20B and 20C. The control information 84G configured in this way is handled in the same way as the control information 84A described in the above embodiment.

[0159] Because the control information 84G includes images 20B and 20C, the imaging device 12 can grasp the amount of change from the "original state" to the "final finish," and can directly learn how much to change the exposure and color tone to achieve the ideal form. As a result, the imaging device 12 can obtain an image close to the ideal from the start in subsequent imaging. Furthermore, even if the imaging device 12 cannot grasp the order of multiple processes included in the image processing 29, the combination of multiple processes included in the image processing 29, and / or the multiple processing parameters 82, the imaging device 12 can estimate the order of multiple processes included in the image processing 29, the combination of multiple processes included in the image processing 29, and / or the optimal values ​​of the multiple processing parameters 82 from the difference between image 20B and image 20C (i.e., the difference between the original image and the processed image). In addition, there is the advantage that the settings of the imaging device 12 can be flexibly corrected regardless of the manufacturer or software. Note that image 20C is an example of a "processed image" according to this disclosure.

[0160] In the example shown in Figure 15, the control information 84G includes both image 20B and image 20C. However, this is merely one example, and the control information 84G does not necessarily have to include image 20B.

[0161] In the example shown in Figure 16, control information 84H is generated by the processor 60. Control information 84H differs from control information 84G in that it includes sequence information 90. Sequence information 90 is information indicating the order of multiple processes included in the image processing 29. Control information 84H configured in this way is handled in the same way as control information 84A described in the above embodiment.

[0162] Because the control information 84H includes sequence information 90, the imaging device 12 can comprehensively analyze the image content and the order of image processing 29, for example, determining whether to "prioritize ISO if brightness is to be adjusted first" or "the sunset mode is appropriate for white balance." Furthermore, the imaging device 12 can understand what settings are necessary at each stage to obtain the final result. Moreover, it can get as close as possible to the desired result for the user 15 at the time of imaging.

[0163] In the example shown in Figure 16, the control information 84H includes images 20B and 20C, but this is merely one example, and the control information 84H does not necessarily have to include images 20B and / or 20C.

[0164] In the example shown in Figure 17, control information 84I is generated by the processor 60. Control information 84I differs from control information 84H in that it includes support information 92. Support information 92 is information that supports imaging by the imaging device 12. Control information 84I configured in this way is handled in the same way as control information 84A described in the above embodiment.

[0165] An example of support information 92 is text information and / or audio information that supports imaging by the imaging device 12. Because the control information 84I includes support information 92, for example, imaging support guides such as "Next time, it would be good to set the exposure to +0.5" or "It would be good to set the white balance for sunset" are displayed on the screen of the display 36B of the imaging device 12. This makes it possible for the user 15 to check and adjust the imaging device parameters more intuitively. Furthermore, even beginners who are not familiar with the settings of the imaging device 12 can easily reproduce advanced imaging settings by simply following the support information 92. As a result, it is possible to achieve both improved imaging quality and improved imaging skills for the user 15.

[0166] In the example shown in Figure 17, the control information 84I is shown to include images 20B, 20C, and sequence information 90. However, this is merely one example, and the control information 84I does not necessarily have to include images 20B, 20C, and / or sequence information 90.

[0167] Furthermore, the control information 84A to 84I (hereinafter referred to as "control information" without symbols unless it is necessary to distinguish between them) may include information regarding imaging device parameters that eliminate the need for image processing 29 (for example, information that can identify imaging conditions 72, development system parameters 74, and / or effect system parameters 76). In this way, for example, the imaging device 12 will perform the brightness adjustment that was previously done each time by the image processing software of the information processing device 14, and an image of equivalent quality can be obtained in the end without subsequent editing. This greatly improves the work efficiency of the user 15. In addition, the imaging device 12 can learn which parameters to set so that the processing performed by the image processing software of the information processing device 14 becomes almost unnecessary. As a result, there is an advantage in that high-quality photographs can be obtained on-site and immediately used for SNS or business purposes.

[0168] Furthermore, instead of completely eliminating the image processing 29 in the information processing device 14, it is sufficient to reduce some of the processing. Also, for example, instead of frequently performing the process of blurring the background of the subject in the image processing 29 of the information processing device 14, the imaging device 12 may learn the "editing result that requires background blurring" and automatically set imaging conditions using a small F-number for similar scenes in the future. As a result, the subsequent image processing 29 for blurring the background becomes unnecessary or significantly reduced, and the user 15 can achieve the desired expression more efficiently.

[0169] Furthermore, a condition may be set to include image 20B in the control information. For example, in this case, the condition for including image 20B in the control information may be determined based on the content of image 20B and / or the content of the image processing 29 performed on image 20B. Furthermore, a condition may be set to include image 20C in the control information. For example, in this case, the condition for including image 20C in the control information may be determined based on the content of image 20C and / or the content of the image processing 29 performed on image 20B.

[0170] In this way, it is not necessary to send a high-resolution image from the information processing device 14 to the imaging device 12 every time, and it is possible to flexibly decide, for example, to include image 20B and / or image 20C in the control information only when the exposure compensation is large. As a result, the communication load and processing load of the system can be reduced. In addition, for example, by including image 20B and / or image 20C in the control information only when a large color correction is performed, and not including image 20B and image 20C in the control information when only fine adjustments to the color correction are made, the control information can be made lighter.

[0171] Furthermore, the control information may be applied to imaging devices other than imaging device 12. This allows the user's settings and imaging tendencies accumulated with one imaging device to be continuously used with another new imaging device. In other words, when purchasing or replacing an imaging device, the user's settings and imaging tendencies accumulated with imaging device 12 can be transferred. In addition, it may be possible to share basic imaging settings and preferences even when switching between manufacturers. Moreover, the user can significantly reduce the effort required to change models.

[0172] In the above embodiment, an example was given in which imaging device parameters are derived from control information 84A, but this is merely one example. For example, when the imaging device 12 sets one of several imaging modes (e.g., landscape mode, portrait mode, and night view mode, etc.) according to instructions given from the outside and performs imaging under the set imaging mode, one of the multiple imaging modes may be derived from the control information 84A, and the derived imaging mode may be forcibly set for the imaging device 12.

[0173] To achieve this, for example, as shown in Figure 18, the NVM 48 stores imaging mode setting information 94 and an imaging mode derivation table 96. The imaging mode setting information 94 is information indicating the imaging mode currently set for the imaging device 12. The imaging mode derivation table 96 is a table that defines the correspondence between control information 84A and imaging modes. In the example shown in Figure 18, the processor 46 obtains control information 84A from the information processing device 14. Then, the processor 46 refers to the imaging mode derivation table 96 and converts the control information 84A obtained from the information processing device 14 into the corresponding imaging mode (portrait mode 98 in the example shown in Figure 18). That is, the processor 46 derives the imaging mode corresponding to the control information 84A obtained from the information processing device 14 from the imaging mode derivation table 96. In the example shown in Figure 18, an example of how portrait mode 98 is derived is shown. The processor 46 sets the imaging mode of the imaging device 12 to portrait mode 98 derived from the imaging mode derivation table 96, and updates the imaging mode setting information 94 stored in the NVM 48 by rewriting its contents with information indicating portrait mode 98.

[0174] By controlling the imaging mode using control information 84A, the imaging device 12 learns the contents of the image processing 29 of the information processing device 14 and can automatically switch between multiple operating modes such as landscape mode, portrait mode, and night scene mode. For example, if user 15 frequently performs "correction to make the blue sky more vivid" in past editing, the imaging device 12 will prioritize "landscape mode," achieving automatic optimization of the imaging mode. Furthermore, while previously users could only choose from preset modes such as landscape mode or portrait mode determined by the manufacturer, multiple processing parameters 82 used in image processing 29 can be collectively registered as a custom imaging mode in the imaging device 12. In other words, the contents of image processing 29 can be reproduced exactly the next time imaging is performed. As a result, even if user 15 is a beginner, they can easily recall an imaging mode that closely matches their preferred finish with simple operation (for example, one button press), allowing them to use the appropriate settings without hesitation.

[0175] In the example shown in Figure 18, an example of how the imaging mode is derived using the imaging mode derivation table 96 is given. However, this is merely one example, and a generating AI that has learned the combination of control information 84A and imaging mode may generate the imaging mode corresponding to the control information 84A.

[0176] In the above embodiment, an example was given in which image processing 29 is performed and control information 84A is generated within the information processing device 14, but this is merely one example. For example, as shown in Figure 19, the image processing 29 may be performed by the processor 46 within the imaging device 12, and the control information 84A may be generated by the processor 46. The image processing 29 within the imaging device 12 is implemented by image processing software within the imaging device 12.

[0177] By performing image processing 29 within the imaging device 12 in this way, the user 15 can complete the entire process, from imaging and editing to reflecting the next imaging settings, using only the imaging device 12. Therefore, advanced processing becomes possible even in situations where the information processing device 14 is unavailable, such as when the user is away from home. Furthermore, by performing image processing 29 within the imaging device 12 immediately after imaging, the results of the image processing 29 can be immediately reflected in the next imaging settings of the imaging device 12. Therefore, it is also effective in professional settings where real-time performance and immediate on-site confirmation are important (for example, news reporting or sports).

[0178] Furthermore, by ensuring that image processing 29 within the imaging device 12 is performed by image processing software within the imaging device 12, the manufacturer's proprietary image processing software can make maximum use of the hardware characteristics of the imaging device 12 (e.g., sensor characteristics and / or lens correction profiles). This also facilitates integrated management of image processing 29 and imaging device parameters. Moreover, the processing history generated by the image processing software within the imaging device 12 can be reflected in subsequent imaging.

[0179] In the above embodiment, an example of how information generation processing is performed by a computer 54 was described, but this disclosure is not limited thereto, and at least some of the processing included in the information generation processing may be performed by a device provided outside the computer 54. An example of this case will be described below with reference to Figure 20.

[0180] Figure 20 is a conceptual diagram showing an example of the configuration of the processing unit 100. The processing unit 100 is an example of the "processing unit" according to this disclosure. The processing unit 100 differs from the information processing unit 14 described in the above embodiment in that it has an external device 102.

[0181] The external device 102 is connected to the computer 54 via the network 24 so as to be able to communicate with it. An example of the external device 102 is at least one server that directly or indirectly sends and receives data with the computer 54 via the network 24. The external device 102 receives processing execution instructions from the processor 60 of the computer 54 via the network 24. The external device 102 then executes the processing according to the received processing execution instructions and transmits the processing results to the computer 54 via the network 24. In the computer 54, the processor 60 receives the processing results transmitted from the external device 102 via the network 24 and executes processing using the received processing results.

[0182] Examples of processing execution instructions include an instruction to have the external device 102 execute at least a part of the information generation process.

[0183] One example of at least a part of the information generation process (i.e., the process to be executed by the external device 102) is image processing 29 (see Figure 6). In this case, the external device 102 executes the image processing 29 according to the processing execution instructions given from the processor 60 via the network 24, and transmits an image file containing the image 20C as the processing result to the computer 54 via the network 24. In the computer 54, the processor 60 receives the image file containing the image 20C and performs the same processing as in the above embodiment using the received image file containing the image 20C.

[0184] A second example of at least a part of the information generation process (i.e., the process to be executed by the external device 102) is image processing 29 and image file output processing 80 (see Figure 6). In this case, the external device 102 executes the image processing 29 and image file output processing 80 according to the processing execution instructions given from the processor 60 via the network 24, and transmits information to the computer 54 indicating that the execution of the image processing 29 and image file output processing 80 has been completed.

[0185] A third example of at least a part of the information generation process (i.e., the process to be executed by the external device 102) is the process of generating control information (see Figure 6). In this case, the external device 102 generates control information 84A according to the processing execution instructions given from the processor 60 via the network 24, and transmits the control information 84A to the computer 54 as a processing result. In the computer 54, the processor 60 receives the control information 84A and uses the received control information 84A to execute the same process as in the above embodiment.

[0186] A fourth example of at least a part of the information generation process (i.e., the process to be executed by the external device 102) is the process of generating control information and the control information output process 86 (see Figure 6). In this case, the external device 102 executes the process of generating control information and the control information output process 86 in accordance with the processing execution instructions given from the processor 60 via the network 24, and as a result of the processing, transmits information to the computer 54 indicating that the execution of the process of generating control information and the control information output process 86 has been completed.

[0187] The external device 102 may be implemented through cloud computing. Cloud computing is merely one example; the external device 102 may also be implemented through network computing such as fog computing, edge computing, or grid computing.

[0188] In the above embodiment, image 20 is shown as an example of a still image, but this disclosure is also valid for moving images.

[0189] In the above embodiments, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In this case, the processor is configured to work in cooperation with the program to execute the various processes in the above embodiments, and can function as a unit or means in the above embodiments. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0190] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of hardware such as a CPU, MPU, FPGA or other programmable logic device, ASIC or other dedicated circuitry for executing specific processes, GPU, and / or NPU. The type of hardware may also be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, the multiple hardware components may reside in physically separate devices or in the same device. Furthermore, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. Hardware is composed of electrical circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0191] Furthermore, the program may be software such as firmware or microcode. Alternatively, the program may be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform its respective function. The program may also be program code and / or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage). The program may be divided and stored on multiple non-temporary computer-readable media located in physically separate devices. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0192] Furthermore, while the above embodiment illustrates a configuration in which the information generation program 68 is pre-stored in the NVM 62 (i.e., installed), this disclosure is not limited thereto. The information generation program 68 may be provided in a form stored on a storage medium such as a CD-ROM, DVD-ROM, and / or USB memory. Alternatively, the information generation program 68 may be provided in a form that is downloaded from an external device via a network.

[0193] This disclosure covers all program products. Program products include all forms of products for providing programs. For example, program products include programs provided via networks such as the Internet, and non-temporary computer-readable storage media such as CD-ROMs, DVDs, and USB memory sticks on which programs are stored.

[0194] The information generation process described above is merely an example. Therefore, it goes without saying that you may remove unnecessary steps, add new steps, or change the processing order, as long as you do not deviate from the main purpose.

[0195] The descriptions and illustrations presented above are detailed explanations of the parts related to this disclosure and are merely examples of this disclosure. For example, the above explanation of the structure, function, operation, and effect is an example of the structure, function, operation, and effect of the parts related to this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace parts of the descriptions and illustrations presented above, as long as you do not deviate from the spirit of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the parts related to this disclosure, explanations of common technical knowledge, etc., that do not require special explanation to enable the implementation of this disclosure have been omitted from the descriptions and illustrations presented above.

[0196] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0197] The following additional information is disclosed regarding the embodiments described above.

[0198] (Note 1) A processing device comprising a processor, wherein the processor performs image processing on an image obtained by imaging by an imaging device, and generates control information for controlling the imaging based on the content of the image processing.

[0199] (Note 2) The control information described above is the processing device described in Note 1, which includes information regarding the aperture value, shutter speed, and ISO sensitivity of the imaging device.

[0200] (Note 3) The processing apparatus described in Note 1 or Note 2, wherein the above image processing includes a plurality of processes, the control information includes the captured image that is the target of the above image processing, and the order in which the plurality of processes are executed, and the condition for including the captured image in the control information is determined based on the content of the captured image and / or the type of image processing.

[0201] (Note 4) An imaging device that is updated using the control information generated by the processing device described in any one of Notes 1 to 3.

[0202] (Note 5) The imaging apparatus according to Note 4, wherein an allow mode that allows the control of the imaging based on the control information acquired from the processing apparatus is selectively set, and a reject mode that rejects the control of the imaging based on the control information acquired from the processing apparatus is rejected.

Claims

1. A processing device comprising a processor, wherein the processor performs image processing on an image obtained by imaging by an imaging device, and generates control information for controlling the imaging based on the content of the image processing.

2. The processing apparatus according to claim 1, wherein the image processing includes a plurality of processes, and the content of the image processing includes the order and / or combination in which the plurality of processes are performed.

3. The processing apparatus according to claim 1, wherein the image processing includes processing using a generation AI.

4. The processing apparatus according to claim 1, wherein the control information includes support information for assisting the imaging.

5. The processing apparatus according to claim 1, wherein the control information includes information regarding imaging conditions that make the image processing unnecessary.

6. The processing apparatus according to claim 1, wherein the control information is acquired by the imaging apparatus through communication between the imaging apparatus and the processing apparatus.

7. The processing apparatus according to claim 1, wherein the control information includes the captured image that is the target of the image processing.

8. The processing apparatus according to claim 1, wherein the image processing includes a plurality of processes, and the control information includes the captured image that is the target of the image processing and the order in which the plurality of processes are performed.

9. The apparatus according to claim 1, wherein the control information includes the captured image that is the target of the image processing, and the processed image obtained by performing the image processing on the captured image.

10. The processing apparatus according to claim 1, wherein the processor acquires information relating to the imaging and generates the control information based on the content of the image processing and the information relating to the imaging.

11. The processing apparatus according to claim 10, wherein the imaging information includes information regarding the tendencies of the person who performed the imaging.

12. The processing apparatus according to claim 1, wherein the control information is applied to an imaging device different from the imaging device.

13. The processing apparatus according to claim 1, wherein if the imaging device performs imaging based on settings applied to the imaging device without using at least a portion of the control information, the processor generates the control information based on the content of the image processing and information regarding the settings.

14. The processing apparatus according to claim 1, wherein the processor transmits the control information to an external device of the processing apparatus.

15. The processing apparatus according to claim 1, wherein the control information is information used for updating the imaging device.

16. The processing apparatus according to claim 1, wherein the image processing is performed outside the imaging device.

17. The processing apparatus according to claim 16, wherein the image processing is performed by the external image processing software.

18. The processing apparatus according to claim 1, wherein the image processing is performed within the imaging device.

19. The processing apparatus according to claim 18, wherein the image processing is performed by image processing software in the imaging device.

20. An imaging device that is updated using the control information generated by the processing device according to any one of claims 1 to 19.

21. The imaging device according to claim 20, having an imaging mode for performing the imaging, wherein the imaging mode is controlled by the control information.

22. The imaging device according to claim 20, comprising an engine for analyzing a subject or scene, wherein the analysis results by the engine are reflected in the control information.

23. A processing method comprising performing image processing on an image captured by an imaging device, and generating control information for controlling the imaging based on the content of the image processing.

24. A program for causing a computer to perform a process that includes performing image processing on an image captured by an imaging device, and generating control information to control the imaging based on the content of the image processing.

Images