Information processing system, information processing device, information processing method, information processing program, imaging device, imaging device control method, and control program

Abstract

Provided are an information processing system, an information processing device, an information processing method, an information processing program, an imaging device, a method for controlling the imaging device, and a control program with which it is possible to easily match colors in a plurality of imaging devices.　An information processing system comprising an imaging device that operates in a master mode, at least one imaging device that operates in a slave mode, and an information processing device, the master-mode imaging device and the at least one slave-mode imaging device transmitting, to the information processing device, a chart image generated by photographing a specified chart, and the information processing device calculating a color conversion coefficient for the at least one slave-mode imaging device on the basis of the chart image generated by the master-mode imaging device and transmitting the color conversion coefficient to the corresponding imaging device.

Description

[Technical Field]

[0001] The present technology relates to an information processing system, an information processing device, an information processing method, an information processing program, an imaging device, a control method for an imaging device, and a control program. [Background technology]

[0002] Multi-camera systems, which use multiple cameras to shoot footage, are widely used in studio recordings of television programs and live sports broadcasts. In multi-camera systems, images can be generated from any desired angle by switching the camera to match the direction of the production or the movement of the subject.

[0003] In such a multi-camera system, in order to avoid any sense of incongruity when viewing images taken with different cameras, it is necessary to perform color matching processing for each camera to unify the colors of each camera (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-146936 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 does not provide specific details about the difference between the color of a camera that serves as a reference for color matching among multiple cameras and the color of other cameras that are matched to that reference camera, and accurate color matching is not possible unless the difference is known.

[0006] The present technology has been developed in consideration of these points, and aims to provide an information processing system, an information processing device, an information processing method, an information processing program, an imaging device, a control method for an imaging device, and a control program that can easily perform color matching among multiple imaging devices. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a first technique is an imaging device including an imaging device operating in a master mode, at least one imaging device operating in a slave mode, and an information processing device, wherein the imaging device in the master mode and the imaging device in the slave mode photograph a specific chart and generate a chart image, and transmit the chart image to the information processing device, and the information processing device: When any of the chart images is received, a shooting operation invalidation instruction is transmitted to all the imaging devices to invalidate the shooting operation from the user; a color conversion unit that calculates color conversion coefficients for an imaging device in a slave mode based on a chart image generated by an imaging device in a master mode and transmits the color conversion coefficients to a corresponding imaging device, the imaging device comprising: When receiving a shooting operation disable instruction, the device puts itself into a shooting operation disable state, sets a color conversion coefficient for color gamut conversion in the color gamut conversion unit, and then puts itself into a shooting operation enable state. It is an information processing system.

[0008] The second technique is a color conversion coefficient calculation unit that calculates color conversion coefficients for an external imaging device that operates in a slave mode and performs color gamut conversion based on color conversion coefficients, based on a chart image generated by an external imaging device that operates in a master mode and captures a specific chart. When the chart image transmitted from the imaging device is received, a shooting operation invalid instruction for invalidating the shooting operation from the user is transmitted to all the imaging devices. It is an information processing device.

[0009] Furthermore, the third technique calculates color conversion coefficients for an external imaging device operating in slave mode and performing color gamut conversion based on a chart image generated by an external imaging device operating in master mode by photographing a specific chart. When the chart image transmitted from the imaging device is received, the imaging device transmits a shooting operation invalid instruction to all imaging devices to invalidate the shooting operation from the user. It is an information processing method.

[0010] In order to solve the above-mentioned problem, the fourth technique calculates color conversion coefficients for an external imaging device operating in slave mode and performing color gamut conversion based on color conversion coefficients, based on a chart image generated by an external imaging device operating in master mode photographing a specific chart. When the chart image transmitted from the imaging device is received, the imaging device transmits a shooting operation invalid instruction to all imaging devices to invalidate the shooting operation from the user. It is an information processing program that causes a computer to execute the information processing method.

[0011] In order to solve the above-mentioned problems, a fifth technique is a color gamut conversion unit that operates in a master mode or a slave mode, transmits a chart image generated by photographing a specific chart to an information processing device, and when operating in the slave mode, receives color conversion coefficients calculated by the information processing device from the information processing device, sets the color conversion coefficients for color gamut conversion, and performs color gamut conversion based on the color conversion coefficients. When receiving a shooting operation invalid instruction from the user that invalidates the shooting operation, transmitted from the information processing device that has received the chart image, the device puts itself into a shooting operation invalid state, and after setting a color conversion coefficient for color gamut conversion in the color gamut conversion unit, puts itself into a shooting operation valid state. It is an imaging device.

[0012] In order to solve the above-mentioned problems, a sixth technique is a color gamut conversion unit that operates in a master mode or a slave mode, transmits a chart image generated by photographing a specific chart to an information processing device, and when operating in the slave mode, receives color conversion coefficients calculated by the information processing device from the information processing device, sets the color conversion coefficients for color gamut conversion, and performs color gamut conversion based on the color conversion coefficients. When receiving a shooting operation invalid instruction from the user that invalidates the shooting operation, transmitted from the information processing device that has received the chart image, the device puts itself into a shooting operation invalid state, and after setting a color conversion coefficient for color gamut conversion in the color gamut conversion unit, puts itself into a shooting operation valid state. A method for controlling an imaging device.

[0013] In order to solve the above-mentioned problems, the seventh technology is a color gamut conversion unit that operates in a master mode or a slave mode, transmits a chart image generated by photographing a specific chart to an information processing device, and when operating in the slave mode, receives color conversion coefficients calculated by the information processing device from the information processing device, sets the color conversion coefficients for color gamut conversion, and performs color gamut conversion based on the color conversion coefficients. When receiving a shooting operation invalid instruction from the user that invalidates the shooting operation, transmitted from the information processing device that has received the chart image, the device puts itself into a shooting operation invalid state, and after setting a color conversion coefficient for color gamut conversion in the color gamut conversion unit, puts itself into a shooting operation valid state. The control program causes a computer to execute the image capture device control method. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a configuration of an information processing system 10. FIG. [Figure 2] FIG. 1 is a diagram showing the configuration of a chart. [Figure 3] FIG. 1 is a diagram illustrating an example of network connections in an information processing system 10. [Figure 4] 1 is a block diagram showing a configuration of an information processing device 100. FIG. [Figure 5] 1 is a block diagram showing a configuration of functional blocks of an information processing device 100. FIG. [Figure 6] FIG. 2 is a block diagram showing the configuration of an imaging device 200. [Figure 7] 2 is a sequence diagram showing processing in the information processing system 10. FIG. [Figure 8] 2 is a sequence diagram showing processing in the information processing system 10. FIG. [Figure 9] FIG. 10 is an explanatory diagram of color conversion coefficient calculation. [Figure 10] FIG. 10 is an explanatory diagram of color conversion coefficient calculation. [Figure 11] This is the formula for multivariate least squares. [Figure 12] FIG. 2 is a diagram illustrating another exemplary configuration of the information processing device 100. [Figure 13] FIG. 10 is an explanatory diagram of keystone correction. [Figure 14] FIG. 10 is an explanatory diagram of a rotation process. [Figure 15] FIG. 2 is a diagram illustrating an example of a UI for the information processing system 10. [Figure 16] 10 is a flowchart showing a process for adding an imaging device 200. [Figure 17] 10A and 10B are diagrams showing the state of the imaging device 200 by turning on a light. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order. <1. Embodiment> [1-1. Configuration of Information Processing System 10] [1-2. Configuration of information processing device 100] [1-3. Configuration of imaging device 200] [1-4. Processing in Information Processing System 10] [1-5. User Interface] <2. Modifications>

[0016] <1. Embodiment> [1-1. Configuration of Information Processing System 10] The configuration of an information processing system 10 according to the present technology will be described with reference to Fig. 1. The information processing system 10 is configured with an information processing device 100, a first imaging device 200A, a second imaging device 200B, and a third imaging device 200C. Although three imaging devices 200 are shown, the number of imaging devices 200 is not limited and may be any number equal to or greater than two. In the following description, when there is no need to distinguish between the first imaging device 200A, the second imaging device 200B, and the third imaging device 200C, they will be referred to as imaging devices 200.

[0017] The information processing device 100 is configured as a device such as a personal computer, a smartphone, or a tablet terminal, and the information processing device 100 and the imaging device 200 are connected via a network or the like.

[0018] The information processing device 100 calculates color conversion coefficients for performing calibration to match colors in all of the image capturing devices 200.

[0019] The imaging device 200 is used by a user to capture images for recording television programs, live sports broadcasts, and the production of movies and dramas. In this embodiment, the imaging device 200 operates in master mode or slave mode. The master mode is the operating mode of the imaging device 200 that serves as the basis for calculation of color conversion coefficients by the information processing device 100, and the slave mode is the operating mode of all imaging devices 200 other than the imaging device 200 operating in master mode. The information processing device 100 calculates color conversion coefficients for the imaging device 200 operating in slave mode so that the colors of the imaging device 200 operating in slave mode match those of the imaging device 200 operating in master mode. In the following description, an imaging device operating in master mode may be referred to as a master imaging device, and an imaging device operating in slave mode may be referred to as a slave imaging device.

[0020] The information processing device 100 calculates color conversion coefficients using an image (hereinafter referred to as a chart image) generated by capturing a chart using the imaging device 200. Examples of charts include a Macbeth chart composed of patches of multiple colors, as shown in FIGS. 2A and 2B. As long as the chart is rectangular and the shape of each color patch constituting the chart is also rectangular, any color chart can be used. The more colors a chart has, the more optimal color matching can be achieved for a variety of colors, which is preferable. In this embodiment, the Macbeth chart shown in FIG. 2A is used. The chart may also be an image of a person, landscape, or other subject, as shown in FIG. 2C. When performing color matching for photographing a person, it is preferable to use a captured image of a person as the chart. The Macbeth chart is a chart based on patches, while the image chart is a chart based on pixels. Color matching refers to matching the color gamuts of one imaging device 200, which serves as a reference, with the color gamuts of other imaging devices 200, so that the colors of the captured images are the same or the color difference is less than a predetermined value.

[0021] The master imaging device 200 and the slave imaging device 200 capture images of specific charts to generate chart images. The specific chart for the master imaging device 200 and the specific chart for the slave imaging device 200 may be the same chart as an object, or may be different charts as an object. If the charts are different as objects, one specific chart must be configured with the same patch color, number, and arrangement as the other specific chart. Therefore, the master imaging device 200 and the slave imaging device 200 may exist in the same space and capture the same chart, or the master imaging device 200 and the slave imaging device 200 may exist in separate spaces and capture different charts as objects.

[0022] 3 shows an example of device connections at a shooting location. Basically, all devices need to be connected to a network in order to connect to the information processing device 100. Two typical examples are shown here.

[0023] In the first example, as shown in FIG. 3A, a LAN (Local Area Network) is constructed with each device, and only one device (information processing device 100 in FIG. 3A) serves as the access point to connect to the external Internet (WAN: Wide Area Network). In FIG. 3A, information processing device 100 serves as the access point, but any device may serve as the access point. Connecting devices on-site via LAN has the advantage of enabling high-speed, broadband communication between devices. Furthermore, devices other than the access point only need to be equipped with, for example, WiFi functionality, and each device does not need to individually sign a contract with a WAN communications provider.

[0024] The second example is a configuration in which all devices are directly connected to the external network (WAN), as shown in Figure 3B (WAN configuration example). While this configuration is characterized by its simple connection configuration, it requires that each device has a contract with a WAN communications provider to be able to communicate.

[0025] [1-2. Configuration of information processing device 100] Next, the configuration of the information processing device 100 will be described with reference to Fig. 4. The information processing device 100 comprises a control unit 150, a storage unit 160, an interface 170, an input unit 180, and a display unit 190.

[0026] The control unit 150 is composed of a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc. The CPU executes various processes and issues commands according to programs stored in the ROM, thereby controlling the entire information processing device 100 and each unit.

[0027] The storage unit 160 is, for example, a large-capacity storage medium such as a hard disk or flash memory.

[0028] The interface 170 is an interface between the imaging device 200 and other devices, the Internet, and the like. The interface 170 may include a wired or wireless communication interface. More specifically, the wired or wireless communication interface may include cellular communication such as 3TTE, Wi-Fi, Bluetooth (registered trademark), Near Field Communication (NFC), Ethernet (registered trademark), Serial Digital Interface (SDI), High-Definition Multimedia Interface (HDMI), Universal Serial Bus (USB), and the like. When the imaging device 200 and the information processing device 100 are connected via hardware, the interface 170 may include a connection terminal between the devices or a bus within the device (hereinafter, these may also be referred to as an internal device interface). When the imaging device 200 and the information processing device 100 are implemented as separate devices, the interface 170 may include different types of interfaces for each device. For example, the interface 170 may include both a communication interface and an internal device interface.

[0029] The input unit 180 is used by the user to input various instructions to the information processing device 100. When the user inputs to the input unit 180, a control signal corresponding to the input is generated and supplied to the control unit 150. The control unit 150 then performs various processes corresponding to the control signal. The input unit 180 may be a touch panel, physical buttons, a touch screen integrated with the display unit 305, or the like.

[0030] The display unit 190 is a display device configured by, for example, an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an organic EL (Electro Luminescence) panel, etc. The display unit 190 displays a user interface for using the information processing system 10, etc.

[0031] 5, the information processing device 100 is configured to include functional blocks such as a device management unit 101, a receiving unit 102, a color conversion coefficient calculation unit 103, and a transmitting unit 104. The device management unit 101 and the color conversion coefficient calculation unit 103 are functions realized by a control unit 150. Furthermore, the receiving unit 102 and the transmitting unit 104 transmit and receive data and information to and from the imaging device 200 using an interface 170.

[0032] The device management unit 101 performs operations such as adding / deleting registration of the image capture device 200 that constitutes the information processing system 10, setting the image capture device 200 to operate in master mode / slave mode, and setting automatic calibration on / off.

[0033] The receiving unit 102 receives the chart image transmitted from the imaging device 200 via the interface 170. The received chart image is supplied to the color conversion coefficient calculation unit 103.

[0034] The color conversion coefficient calculation unit 103 calculates color conversion coefficients based on multiple chart images received from multiple image capture devices 200. The color conversion coefficient calculation unit 103 calculates color conversion coefficients for each slave image capture device 200. Therefore, the color conversion coefficients will be different for each slave image capture device 200. If the color gamut conversion unit 204 in the image capture device 200 is configured with a matrix circuit, the color conversion coefficients must be calculated as matrix coefficients. Also, if the color conversion coefficients are configured with a 3D LUT circuit, the color conversion coefficients must be calculated as a 3D LUT. Details of the calculation of the color conversion coefficients will be described later.

[0035] The transmission unit 104 performs processing to transmit the color conversion coefficients calculated by the color conversion coefficient calculation unit 103 to the slave image capture device 200 via the interface 170 .

[0036] The information processing device 100 is configured as described above. Note that the processing in the information processing device 100 may be realized by executing a program, and the execution of the program may cause a personal computer, a tablet terminal, a smartphone, a server device, or the like to have the functions of the information processing device 100. The program may be pre-installed on the information processing device 100, or may be distributed by download or on a storage medium, and installed by the user.

[0037] [1-3. Configuration of imaging device 200] The configuration of the imaging device 200 will be described with reference to Fig. 6. The imaging device 200 is configured to include an imaging unit 201, a correction / white balance processing unit 202, a color separation unit 203, a color gamut conversion unit 204, a Log conversion unit 205, a Look processing unit 206, a display unit 207, an interface 208, a control unit 209, an operation unit 210, and a subject detection unit 211.

[0038] The imaging unit 201 includes a lens, an imaging element that photoelectrically converts incident light from a subject obtained through the lens into an electric charge and outputs an imaging signal, a lens driver that drives the lens, a processing unit that performs A / D (Analog / Digital) conversion on the imaging signal to generate image data, etc. The imaging element may be a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc.

[0039] The image sensor may be a single-chip Bayer array type, a three-chip type, or a single-chip type with an array other than Bayer, and the processing after the RGB image is generated is the same in all cases.

[0040] A correction / white balance processing unit 202 performs optical correction processing and white balance processing on the image data.

[0041] The color separation unit 203 performs, for example, demosaic processing as a color separation process on the image data that has been subjected to white balance processing. Note that the color separation unit 203 is not an essential component, and is not required if the image sensor does not have a Bayer array.

[0042] The color gamut conversion unit 204 performs processing to convert the color gamut of RGB image data using color conversion coefficients calculated by the information processing device 100 or color conversion coefficients that the information processing device 100 has in advance.

[0043] A color gamut is a definition of how colors are represented. Even the same color will have different RGB values ​​if the color gamut differs. There are various types of color gamuts, such as BT.709 for SDR (Standard Dynamic Range), BT.2020 for HDR (High Dynamic Range), DCI-P3 for cinema projectors, and S-Gamut3 for certain manufacturers' log formats. Furthermore, image sensors also have their own color gamuts (sensor gamuts) depending on their color filter characteristics. For example, if one image capture device 200 has color gamut A and the other image capture device 200 has color gamut B, the colors will match if both are converted to a different color gamut C. Alternatively, the color gamut of the image capture device 200 with color gamut A may be converted to color gamut B. However, even if the color gamuts of one image capture device 200 and the other image capture device 200 are converted to the same color gamut, the colors may not match completely due to differences in the spectral characteristics of the image capture devices. Ideally, when a color on that color gamut is represented by converting it into the same color gamut, the RGB values ​​will be the same in one imaging device 200 and the other imaging device 200. However, in reality, there are cases where the colors match perfectly (the difference in RGB signal values ​​is 0), and there are also cases where the colors do not match and are misaligned. This technology can solve such problems and match colors.

[0044] When the imaging device 200 is in master mode, the color gamut conversion unit 204 performs standardization processing to convert the color gamut of the imaging device to a standard color gamut (such as Rec. 709 or S-Gamut3.cine). On the other hand, when the imaging device 200 is in slave mode, the color gamut conversion unit 204 performs standardization processing to convert the color gamut of the imaging device to a standard color gamut, and also sets color conversion coefficients calculated by the information processing device 100 for color gamut conversion and performs color gamut conversion using these color conversion coefficients. This allows color matching in the standard color gamut between the imaging devices 200 before Look processing by the Look processing unit 206. Standardization means standardizing (or sharing) specifications to a standard so that colors can be processed appropriately even among different devices that handle images and videos (including not only imaging devices but also monitors as display devices, editing software, etc.).

[0045] If the color gamut conversion unit 204 is configured with a matrix circuit, the color conversion coefficients must be calculated as matrix coefficients. Also, if the color conversion coefficients are configured with a 3D LUT circuit, the color conversion coefficients must be calculated as a 3D LUT.

[0046] A Log conversion unit 205 converts the linear image data from the color gamut of the imaging device into a standard Log format (such as S-Log3) after the color gamut conversion by the color gamut conversion unit 204. The converted image data may be output to the outside via an SDI (Serial Digital Interface) or the like.

[0047] The recording format of the image data of the imaging device 200 itself is not limited to encoded Log material, but may also be scene-referred RAW material.

[0048] It is also possible to perform look processing on linear image data instead of Log format image data and output linear image data. Even with linear image data, gamma (= logarithmic (Log characteristics)) may be applied when viewing the transmission signal. For example, it is possible to apply gamma to linear image data before transmission, and then apply inverse gamma at the display device (for bandwidth compression on the transmission path, etc.). This gamma-applied Log for transmission is a different Log (gamma) from the Log described above.

[0049] The Look processing unit 206 performs Look processing on the image data after color gamut conversion by the color gamut conversion unit 204. The Look processing unit 206 is implemented, for example, using a 3D LUT circuit, a 1D LUT circuit, or a combination of a 1D LUT circuit and a matrix. A Look is a way of expressing color and gradation, and can be thought of as a visual style that reflects the intentions of the person creating the image / video work. When producing works such as movies, it is common to unify and maintain a consistent Look within the work. It is also necessary to optimize the work for each display device on which it will be viewed. This is because the brightness and color range (color gamut) that can be expressed differ depending on the display device, and the same Look must be realized within the range that can be expressed.

[0050] The Look processing unit 206 performs Look processing on image data based on Look data acquired from a Look data providing system configured as a cloud system or preset Look data built into the main body of the imaging device 200. The Look processing unit 206 may also have a CDL adjustment function.

[0051] The Look Data Providing System is a system that provides Look Data to users, manages the accounts and user information of users who use Look Data, registers and manages electronic devices used by users, and manages user-specific device information, Look Data preference information, etc. Using the Look Data Providing System, businesses that provide Look Data can provide users with comprehensive Look management services, including the provision of Look Data.

[0052] Furthermore, by linking Look data to licenses and managing access, the Look data provision system can also create license agreements with users and a billing system for collecting fees from users. For example, it is possible to create a business model in which people who use Look data (such as photographers or creators performing post-production) are charged, the company that created the Look data (such as a production company) receives a usage fee, and an external cloud system operator receives a portion of the usage fee. Furthermore, it is possible to set up multiple pricing plans based on the number and type of Look data, and create a service in which users can use different Look data depending on the pricing plan they select.

[0053] The Look component data included in the Look data is in a 3D LUT format (such as a .cube file) or 1D LUT format that is applied to Log material (shot in a format such as S-Log3 or S-Gamut3). 1D LUTs are common to both RGB, while 3D LUTs apply different 1D LUTs to each RGB. Log material is a format that faithfully records the optical signals of a subject, and its specifications are generally standardized between imaging devices to match the gradation and color gamut. The Look component data is applied to that Log material. By using 3D LUT data that is applied to standardized Log material, the same Look can be reproduced by applying the same Look data to image data shot in that Log material format, regardless of the imaging device 200 used, regardless of the manufacturer, model, version, etc.

[0054] After the color gamut conversion, the Look processing unit 206 performs Look processing. However, since color matching is performed by color gamut conversion using color conversion coefficients between the master imaging device 200 and the slave imaging device 200, the same results can be obtained by setting the same Look to each Look processing unit 206.

[0055] The display unit 207 is an EVF (Electronic View Finder), a display, or the like that displays image data that has been subjected to Look processing by the Look processing unit 206, a through image, saved image / video data, a GUI (Graphical User Interface), etc. The display unit 207 may be configured, for example, with an LCD, a PDP, or an organic EL panel. The imaging device 200 also has a function of displaying unique identification information (e.g., a QR code (registered trademark)) that identifies the device on the display unit 207. The identification information is used to register and connect the imaging device 200 to the information processing device 100 as a device that constitutes the information processing system 10.

[0056] The identification information may include manufacturer information, device model information, device version information, device model number information, the IP address used by the device, and type information of the display device provided in the device.

[0057] Manufacturer information is information such as the name of a manufacturer that identifies a manufacturer that manufactures and sells the imaging device 200. Device model information is information that indicates the type of imaging device 200. Device version information is information that indicates the version when multiple versions exist for the same model of imaging device 200. It is common for products to be sold after defects are removed or new functions are added, resulting in different versions of the product being sold. Versions indicate such chronological differences. Device model number information is a model number assigned to each model of imaging device 200 to identify it. An IP address is a number that identifies the imaging device 200 on an IP network. Display device type information is information that indicates the type (display method, etc.) of the display unit 207 included in the imaging device 200.

[0058] The display unit 207 displays images based on a predetermined display format. Examples of display formats include Rec. 709 and HLG (Hybrid Log Gamma). These display formats specify various requirements such as resolution, frame rate, color gamut, and expected brightness of the display device. HLG stands for a hybrid of Rec. 709 and Log. Some image capture devices 200 have display units 207 that display images in Rec. 709, while others display images in HLG, which varies depending on the manufacturer, model, version, etc. of the image capture device 200.

[0059] The interface 208 is an interface between the information processing device 100, a network, etc., and is the same as that provided in the information processing device 100. The imaging device 200 is connected to the information processing device 100 via the interface 208 and the network, and can transmit a chart image to the information processing device 100 and further receive color conversion coefficients transmitted from the information processing device 100.

[0060] The control unit 209 is composed of a CPU, RAM, ROM, etc. The CPU executes various processes according to programs stored in the ROM and issues commands to control the entire imaging device 200 and each unit.

[0061] The operation unit 210 includes a release button, a touch panel, and the like that are operated by the user to give instructions for shooting.

[0062] The subject detection unit 211 performs processing to detect the chart using a known subject detection function. The chart detection result is supplied to the control unit 209, and under the control of the control unit 209, the imaging unit 201 captures an image of the chart and generates a chart image.

[0063] Although details will be described later, the control unit 209 puts the imaging device 200 into a state in which a user's imaging operation on the operation unit 210 is disabled in response to a shooting operation disable instruction from the information processing device 100. Also, the control unit 209 controls switching of the imaging device 200 from a shooting operation disabled state to a shooting operation enabled state in response to a shooting operation enable instruction from the information processing device 100. Furthermore, the control unit 209 also controls on / off of automatic calibration.

[0064] Although not shown, the imaging device 200 also includes a storage unit for storing image data, look data, and the like.

[0065] The imaging device 200 is configured as described above. The imaging device 200 may be a device specialized for camera functions, such as a digital camera, a single-lens reflex camera, a camcorder, a commercial camera, or professional imaging equipment, or may be a smartphone, tablet terminal, or wearable device equipped with a camera function. The processing in the imaging device 200 may be realized by executing a program. The program may be pre-installed in the imaging device 200, or may be distributed by download or storage medium, and installed by the user.

[0066] [1-4. Processing in Information Processing System 10] The processing in the information processing system 10 will be described with reference to Figures 7 and 8. For convenience of explanation, Figures 7 and 8 show that processing is performed in the order of first imaging device 200A, second imaging device 200B, and third imaging device 200C, but this does not necessarily mean that processing in first imaging device 200A is performed first; processing in second imaging device 200B or third imaging device 200C may be performed first, or processing in all imaging devices 200 may be performed approximately simultaneously. The order in which imaging devices 200 perform processing is not limited to any particular order.

[0067] First, in step S101, the device management unit 101 of the information processing device 100 sets one of the registered imaging devices 200 as the imaging device 200 that operates in master mode, and sets the others as imaging devices 200 that operate in slave mode.

[0068] The device management unit 101 can use various methods to set one of the registered imaging devices 200 as the imaging device 200 that operates in master mode.

[0069] The device management unit 101 can set an imaging device 200 to operate in master mode based on the respective positions of the multiple imaging devices 200. For example, the imaging device 200 located substantially in front of the subject to be photographed after color matching is set to master mode. When setting an imaging device 200 to operate in master mode based on the position of the imaging device 200 in this way, the information processing device 100 needs to acquire in advance information about the positional relationship between the imaging devices 200 under its management and the subject.

[0070] Furthermore, the device management unit 101 can set an imaging device 200 to operate in master mode based on the performance of the imaging device 200. For example, based on whether the imaging device 200 supports HDR or SDR, an imaging device 200 that supports SDR is set to master mode. This is because HDR can display a wider range of brightness and contrast than SDR, and an SDR imaging device 200 cannot reproduce the color gamut of HDR, making it difficult to match colors to an HDR imaging device 200. When determining an imaging device 200 to be set to master mode based on the performance of the imaging device 200 in this way, the information processing device 100 needs to acquire performance information of the imaging device 200 from the imaging device 200 in advance.

[0071] Furthermore, if it is desired to match the colors of all of the imaging devices 200 rather than matching the colors of one specific imaging device 200 with the colors of the other imaging devices 200, any imaging device 200 may be in master mode, and therefore the device management unit 101 may randomly determine which imaging device 200 from the multiple imaging devices 200 is to be in master mode.

[0072] Furthermore, the device management unit 101 may determine the image capture device 200 to be set to the master mode based on an input from the user.

[0073] In the examples of FIGS. 7 and 8, the first imaging device 200A is set to the master mode, and the second imaging device 200B and the third imaging device 200C are set to the slave mode.

[0074] As shown in step S102, the information processing device 100 transmits a notification to the first imaging device 200A, which has been determined to be in master mode, to operate in master mode. Furthermore, as shown in step S103, the information processing device 100 transmits a notification to the second imaging device 200B, which has been determined to be in slave mode, to operate in slave mode. Furthermore, as shown in step S104, the information processing device 100 transmits a notification to the third imaging device 200C, which has been determined to be in slave mode, to operate in slave mode. Then, in step S105, the first imaging device 200A starts operating in master mode, in step S106, the second imaging device 200B starts operating in slave mode, and in step S107, the third imaging device 200C starts operating in slave mode. The first imaging device 200A in master mode must transmit a notification to the information processing device 100 to inform it that it will be operating in master mode.

[0075] Next, as shown in step S108, when the information processing device 100 turns on automatic calibration, in steps S109, S110, and S111, it sends notifications to the first imaging device 200A, the second imaging device 200B, and the third imaging device 200C to turn on automatic calibration, respectively.

[0076] Then, in steps S112, S113, and S114, the first imaging device 200A, the second imaging device 200B, and the third imaging device 200C each turn on auto-calibration. When auto-calibration is turned on, the generation of a chart image by the imaging device 200, the calculation of color conversion coefficients by the information processing device 100, and the setting of the color conversion coefficients in the imaging device 200 are all performed automatically. Note that the first imaging device 200A in master mode, and the second imaging device 200B and the third imaging device 200C in slave mode each need to send a notification to the information processing device 100 to notify them that their own auto-calibration has been turned on. However, if sending a chart image to the information processing device 100 can serve as an alternative, sending a notification is not necessary.

[0077] Next, as shown in step S115, when the first imaging device 200A detects the chart, it photographs the chart to generate a chart image, and in step S116 it transmits the chart image to the information processing device 100. Then, in step S117, the information processing device 100 receives the chart image transmitted from the first imaging device 200A.

[0078] When the information processing device 100 receives a chart image from any one of the imaging devices 200, it transmits a shooting operation invalidation instruction to the first imaging device 200A, the second imaging device 200B, and the third imaging device 200C, respectively, as shown in steps S118, S119, and S120.

[0079] 7, the information processing device 100 transmits a shooting operation invalid instruction to all of the imaging devices 200 upon receiving a chart image from the first imaging device 200A in master mode, which is merely an example for the sake of convenience of explanation. Regardless of whether the information processing device 100 is in master mode or slave mode, upon receiving a chart image from any one of the imaging devices 200, the information processing device 100 transmits a shooting operation invalid instruction to all of the imaging devices 200.

[0080] In this way, the information processing device 100 does not transmit a shooting operation invalid instruction when it has completed receiving chart images from all of the imaging devices 200, but transmits a shooting operation invalid instruction to all of the imaging devices 200 upon receiving a chart image from any one of the imaging devices 200. This is because calibration begins when a chart image is received from any one of the imaging devices 200, and shooting operations from the user should not be accepted until color conversion coefficients for color gamut conversion have been set in all of the slave imaging devices 200. If a shooting operation were accepted, shooting would occur in a state where color conversion coefficients have not been set and color matching has not been achieved. Note that, because the shooting operation invalid state invalidates shooting operations from the user, when the imaging device 200 detects a chart, it can still capture the chart and generate a chart image.

[0081] Then, as shown in step S121, the first imaging device 200A places itself in a shooting operation disabled state. Similarly, as shown in step S122, the second imaging device 200B also places itself in a shooting operation disabled state. Furthermore, as shown in step S123, the third imaging device 200C also places itself in a shooting operation disabled state. The shooting operation disabled state is a state in which no shooting is performed even if the user performs an operation to instruct shooting, such as pressing the release button. The first imaging device 200A in master mode, and the second and third imaging devices 200B and 200C in slave mode each need to transmit a notification to the information processing device 100 to notify them that they have entered a shooting operation disabled state.

[0082] As shown in step S124, when the second imaging device 200B detects a chart, it captures the chart to generate a chart image, and in step S125 transmits the chart image to the information processing device 100. Then, in step S126, the information processing device 100 receives the chart image transmitted from the second imaging device 200B.

[0083] Furthermore, as shown in step S127, when the third imaging device 200C detects a chart, it photographs the chart to generate a chart image, and in step S128 transmits the chart image to the information processing device 100. Then, in step S129, the information processing device 100 receives the chart image transmitted from the third imaging device 200C.

[0084] 8, the second and third imaging devices 200B and 200C generate chart images and transmit them to the information processing device 100 after entering the shooting operation disabled state, but the processes are not necessarily performed in this order. This is because FIG. 7 is described using an example in which the information processing device 100 transmits a shooting operation disabled instruction to all imaging devices 200 upon receiving the chart image transmitted from the first imaging device 200A. Therefore, when the second imaging device 200B transmits a chart image to the information processing device 100 first, the second imaging device 200B subsequently enters the shooting operation disabled state. Also, when the third imaging device 200C transmits a chart image to the information processing device 100 first, the third imaging device 200C subsequently enters the shooting operation disabled state.

[0085] When the information processing device 100 receives the chart images from all the imaging devices 200, it then calculates color conversion coefficients as shown in step S130. Note that when the information processing device 100 receives at least a chart image from the master imaging device 200 and a chart image from one slave imaging device 200, it may calculate color conversion coefficients for that one slave imaging device 200. In this case, every time the information processing device 100 receives a chart image from another slave imaging device 200, it calculates color conversion coefficients for the other slave imaging device 200.

[0086] Here, we will explain how to calculate the color conversion coefficients. Because the chart image contains noise components, the average pixel value for each patch that makes up the chart is first calculated as the representative value for that patch. For example, the color of a white patch is calculated as (R, G, B) = (240, 240, 240).

[0087] The information processing device 100 uses the chart image of the first imaging device 200A in master mode as a reference and calculates color conversion coefficients optimized for the second imaging device 200B so that the colors of the chart image of the second imaging device 200B in slave mode match the colors of the chart image of the first imaging device 200A. The information processing device 100 also uses the chart image of the first imaging device 200A as a reference and calculates color conversion coefficients optimized for the third imaging device 200C so that the colors of the chart image of the third imaging device 200C in slave mode match the colors of the chart image of the first imaging device 200A. That is, the information processing device 100 uses the chart image of the first imaging device 200A in master mode as a reference and calculates color conversion coefficients for each slave imaging device 200.

[0088] The matrix coefficients as color conversion coefficients are calculated, for example, by using the least squares method so that the difference between the colors of the patches that make up the chart is minimized.

[0089] 9 and 10 are explanatory diagrams of color conversion coefficient calculation. Here, calculation of color conversion coefficients for the second imaging device 200B, which is the slave, is explained. For convenience of explanation, four patches ((1), (2), (3), and (4)) of the multiple patches that make up the chart are extracted and shown in FIGS. 9 and 10.

[0090] 9, when the master first imaging device 200A captures an image of a chart, it performs predetermined image processing such as color separation processing to generate an RGB chart image, and the color gamut conversion unit 204 performs color gamut conversion on the chart image using color conversion coefficients. The color conversion coefficients in the master first imaging device 200A are designated as color conversion coefficients A.

[0091] The chart image before color conversion of the first master imaging device 200A is defined by a 1×3 determinant using three variables (R, G, B) as shown in Equation 1 below.

[0092] [Formula 1] TIFF0007757979000001.tif17125

[0093] Furthermore, the chart image after color conversion of the first master imaging device 200A is defined by a 1×3 determinant using three variables (R, G, B) as shown in Equation 2 below.

[0094] [Formula 2] TIFF0007757979000002.tif17126

[0095] 10, in the second slave imaging device 200B, similarly, when a chart is photographed, predetermined image processing such as color separation processing is performed to generate an RGB chart image, and the color gamut conversion unit 204 performs color gamut conversion on the chart image using color conversion coefficients. The color conversion coefficients in the second slave imaging device 200B are designated as color conversion coefficients B.

[0096] The chart image before color conversion of the second slave imaging device 200B is defined by a 1×3 determinant using three variables (R, G, B), as shown in Equation 3 below.

[0097] [Formula 3] TIFF0007757979000003.tif17125

[0098] Furthermore, the chart image before color conversion of the second slave imaging device 200B is defined by a 1×3 determinant using three variables (R, G, B), as shown in Equation 4 below.

[0099] [Formula 4] TIFF0007757979000004.tif17126

[0100] In this embodiment, the color conversion coefficients are set as matrix coefficients in a 3x3 determinant. For three variables (R, G, B), the color conversion coefficients are calculated from three variables (a, b, c) for calculating R, three variables (d, e, f) for calculating B, and three variables (g, h, i) for calculating G, using the following formula 5: It is defined as follows.

[0101] [Formula 5] TIFF0007757979000005.tif17125

[0102] Furthermore, from the color conversion coefficients of Equation 5, the color conversion coefficient A for the first image capturing device 200A, which is the master, is defined as shown in Equation 6 below.

[0103] [Formula 6] TIFF0007757979000006.tif17125

[0104] Furthermore, from the color conversion coefficients of Equation 5, the color conversion coefficient B for the second slave image capturing device 200B is defined as shown in Equation 7 below.

[0105] [Formula 7] TIFF0007757979000007.tif17125

[0106] The colors of the patches (1), (2), (3), and (4) of the chart image after color conversion in the master first imaging device 200A are expressed by Equations 8-1, 8-2, 8-3, and 8-4 in Equation 8 below using the chart image before color conversion and the color conversion coefficient A.

[0107] [Formula 8] TIFF0007757979000008.tif77125

[0108] Furthermore, the colors of the patches (1), (2), (3), and (4) of the chart image after color conversion in the slave second imaging device 200B are expressed by Equations 9-1, 9-2, 9-3, and 9-4 in the following Equation 9 using the chart image before color conversion and a color conversion coefficient B.

[0109] [Formula 9] TIFF0007757979000009.tif77125

[0110] If the color conversion coefficient B for the second slave imaging device 200B is not set appropriately, the colors in the images after color conversion processing will differ between the first master imaging device 200A and the second slave imaging device 200B. Therefore, the information processing device 100 uses the chart image after color gamut conversion of the first master imaging device 200A as a reference and calculates the color conversion coefficient B for the second slave imaging device 200B so that the colors of the chart image after color gamut conversion of the second slave imaging device 200B match the reference.

[0111] The color conversion coefficient B is calculated by using the least squares method to find common color conversion coefficients (aB to iB) that minimize the sum of squared differences between the colors of the chart image after color gamut conversion of the first imaging device 200A and the colors of the chart image after color gamut conversion of the second imaging device 200B, using Equations 8 and 9. More specifically, for three variables (R, G, B), calculations are performed using the multivariate least squares method for three variables (a, b, c) for calculating R, three variables (d, e, f) for calculating B, and three variables (g, h, i) for calculating G.

[0112] The calculation formula for the multivariate least squares method is shown in Figure 11. For example, the three variables (a, b, c) used to calculate R of the color conversion coefficient B can be calculated using the multivariate least squares method by defining the necessary variables as shown below.

[0113] x=(R B , G B , B B )···n(3 variables) xl: x1~x4 (data before color conversion for the four colors of patches (1), (2), (3), and (4)) yl:R A 1'~ R A 4' (Four color data of the chart image of the master imaging device 200 (reference)) am: a1=a, a2=b, a3=c, m=3 (number of coefficients to be found) gm:g1~g3 RB'=a B ×R B +b B ×B B +cB ×G B

[0114] In this way, the color conversion coefficient B is calculated. The color conversion coefficient for the third imaging device 200C can be calculated in the same way, using the chart image of the first imaging device 200A as a reference. Note that, since all of the above variables except am are known, am = coefficient can be found by matrix calculation.

[0115] As described above, the color conversion coefficients may be not only matrix coefficients but also 3D LUTs, etc. When calculating the color conversion coefficients as a 3D LUT, table data for the 3D LUT is calculated for each slave imaging device 200 so that the color difference with the master imaging device 200 is minimized.

[0116] Returning to the explanation of Figure 8, once the information processing device 100 has calculated the color conversion coefficients, it transmits the color conversion coefficients to the second slave imaging device 200B in step S131, and transmits the color conversion coefficients to the third slave imaging device 200C in step S132. The color conversion coefficients are calculated and optimized for each imaging device 200, and the color conversion coefficients transmitted to the second imaging device 200B and the third imaging device 200C are different.

[0117] Upon receiving the color conversion coefficients, the second imaging device 200B sets the color conversion coefficients for color gamut conversion in the color gamut conversion unit 204 as shown in step S133. Once the setting of the color conversion coefficients is completed, the second imaging device 200B transmits a setting completion notification to the information processing device 100 in step S134.

[0118] Furthermore, upon receiving the color conversion coefficients, the third imaging device 200C sets the color conversion coefficients for color gamut conversion in the color gamut conversion unit 204 as shown in step S135. Once the setting of the color conversion coefficients is completed, the third imaging device 200C transmits a setting completion notification to the information processing device 100 in step S136.

[0119] When the information processing device 100 receives a setting completion notification from the second imaging device 200B and the third imaging device 200C as shown in step S137, it sends instructions to enable shooting operations to the first imaging device 200A, the second imaging device 200B, and the third imaging device 200C, respectively, in steps S138, S139, and S140.

[0120] Then, in step S141, the first imaging device 200A places itself in a photographing operation enabled state, in step S142 the second imaging device 200B places itself in a photographing operation enabled state, and in step S143 the third imaging device 200C places itself in a photographing operation enabled state, thereby enabling the second imaging device 200B and the third imaging device 200C to perform imaging in a state where the color matches that of the first imaging device 200A.

[0121] Note that the imaging device 200 may enable the imaging operation itself when the setting of the color conversion coefficients is complete, without requiring an instruction to enable the imaging operation from the information processing device 100. However, if multiple imaging devices 200 are to start imaging simultaneously, all imaging devices 200 must know that the setting of the color conversion coefficients has been completed for all imaging devices 200. This is because even if the setting of the color conversion coefficients has been completed for one imaging device 200, the setting of the color conversion coefficients may not have been completed for the other imaging devices 200. Therefore, in such a case, each imaging device 200 notifies the other imaging devices 200 that the setting of the color conversion coefficients has been completed, and starts imaging after confirming that the setting of the color conversion coefficients has been completed for all imaging devices 200.

[0122] Then, in step S144, the information processing device 100 enters a waiting state. The waiting state is a state in which the next calibration is not performed for a predetermined time so that the next calibration does not start unintentionally. For example, if the predetermined time is set to one hour, the next calibration will not start automatically until one hour has passed after the calibration is completed.

[0123] When the set time has elapsed, the information processing device 100 again instructs the imaging device 200 to perform chart detection. If the imaging device 200 detects even one chart and transmits a chart image to the information processing device 100, the information processing system 10 transitions to calibration and again enters the shooting operation disabled state.

[0124] In addition, since it is undesirable for different color conversion coefficients to be set during shooting in the imaging device 200, causing the colors to change, in order to prevent this, it is advisable to not accept any notifications, instructions, information, etc. from the information processing device 100 during shooting.

[0125] In this manner, processing in the information processing system 10 is carried out.

[0126] As shown in FIG. 12, the information processing device 100 may be provided with an image processing unit 105 that performs various image processes on the chart image before calculating the color conversion coefficients.

[0127] The information processing device 100 uses the chart images captured by each imaging device 200 to calculate color conversion coefficients. However, depending on the angle of the imaging device 200, the chart in the chart image may be trapezoidal rather than rectangular, as shown in FIG. 13. If this state is left as it is, it is not possible to accurately compare colors, and it is not possible to calculate color conversion coefficients from color differences. Therefore, the image processing unit 105 performs trapezoidal correction on the chart image to convert it to a rectangle. This makes it possible to accurately compare the chart image of the master imaging device 200 and the chart image of the slave imaging device 200, and calculate color conversion coefficients from color differences.

[0128] Also, when the information processing device 100 receives chart images from the master imaging device 200 and the slave imaging device 200, consider a case where the color difference between each patch in the master chart image and the slave chart image is equal to or greater than a predetermined value, as shown in Fig. 14. A color difference equal to or greater than a predetermined value means, for example, that ΔE (one of the color difference indexes) is 10 or greater.

[0129] In this case, it is determined that the slave chart image is rotated 180 degrees relative to the master chart image because the user accidentally photographed the chart upside down, and the image processing unit 105 performs 180-degree rotation processing to correct the orientation of the slave image. Then, when the color difference for each patch between the master chart image and the rotated slave chart image becomes equal to or less than a predetermined value, color conversion coefficients are calculated.

[0130] In addition, in cases where the chart is square, for example, a chart image may be captured with the chart rotated 90 degrees, so the image processing unit 105 may perform a 90-degree rotation process, and the rotation angle is not limited to 180 degrees.

[0131] [1-5. User Interface] Next, a description will be given of a user interface for using the information processing system 10. This user interface is displayed on a device on which the information processing device 100 operates.

[0132] The user interface shown in FIG. 15 includes an imaging device display 401, a master / slave switching button 402, a delete button 403, an add button 404, an automatic calibration setting button 405, and a wait time setting button 406.

[0133] The imaging device display 401 displays the name of the imaging device 200 that constitutes the information processing system 10 and is connected / registered to the information processing device 100 .

[0134] The master / slave switching button 402 is used by the user to switch the imaging device 200 between operating in master mode and operating in slave mode.

[0135] Of the multiple image capture devices 200, only one operates in master mode, so when an input is made to set any one of the image capture devices 200 to master mode, the other image capture devices 200 are automatically set to slave mode. For example, the master / slave switching button 402 corresponding to the master image capture device 200 is displayed as M, and the master / slave switching button 402 corresponding to the slave image capture device 200 is displayed as S, allowing the user to visually confirm whether the device is in master mode or slave mode.

[0136] The delete button 403 is input when removing any image capturing device 200 from the information processing system 10. When the user inputs an input into the delete button 403, the communication connection between the image capturing device 200 corresponding to the delete button 403 and the information processing device 100 is cut off, and the image capturing device 200 does not become part of the information processing system 10.

[0137] The add button 404 is input when adding a new imaging device 200 to the information processing system 10 and registering the information processing device 100. For example, if the imaging device 200 has a function for displaying device-specific identification information (such as a QR code (registered trademark)) on the display unit 207, the user operates the imaging device 200 to display the identification information. The user then reads the identification information using a camera function (which may be provided in the information processing device 100 or an external camera device) and inputs the information processing device 100 into the add button 404, whereby the information processing device 100 can obtain the identification information and unique network information of the new imaging device 200. The information processing device 100 and the imaging device 200 are then connected, and under the control of the control unit 150 of the information processing device 100, the device management unit 101 determines the identification information of the new imaging device 200 and registers the imaging device 200. In addition to photographing identification information such as a QR code (registered trademark), other methods may be used, such as inputting an ID unique to the imaging device 200 into the information processing device 100, searching for devices connected to a common network, or detecting devices with a wireless network connection function such as Bluetooth (registered trademark).

[0138] The user can switch automatic calibration on and off by inputting an input to the automatic calibration setting button 405. When automatic calibration is on, it can be turned off at any time, and if the user inputs to turn it off, the information processing device 100 will forcibly stop the calibration even if it is in progress. In this case, the most recent color conversion coefficients that were last set will remain set in the imaging device 200.

[0139] In the information processing system 10, it may be possible to configure the information processing system 10 so that calibration cannot be performed again until a predetermined time has elapsed after calibration has been performed. This can be achieved by measuring time using a timing function that is normally provided in the information processing device 100 and the imaging device 200, and by preventing a chart from being photographed or a chart image from being generated even if a chart is detected after calibration has been performed and before the predetermined time has elapsed.

[0140] The wait time setting button 406 is used to set the time after calibration has been performed until calibration can be performed again. If you do not want to change the settings of the color conversion coefficients in the color gamut conversion unit 204, setting an appropriate time can prevent unnecessary calibration from being performed.

[0141] Here, the registration and deletion process of the imaging device 200 performed by the information processing device 100 will be described with reference to the flowchart of FIG.

[0142] First, in step S201, it is determined whether or not there has been an input from the user, and if there has been an input, the process proceeds to step S202 (Yes in step S201). Next, if in step S202 the input from the user is an input to the add button 404, the process proceeds to step S203 (Yes in step S202). Next, in step S203, the device management unit 101 acquires identification information of the image capture device 200. The identification information can be acquired by recognizing a QR code (registered trademark) as described above. Then, in step S204, the image capture device 200 corresponding to the identification information is registered as a device constituting the information processing system 10.

[0143] On the other hand, if the user input is not an input to the add button 404 in step S202, the process proceeds to step S205 (No in step S202). Next, if the user input is an input to the delete button 403 in step S205, the process proceeds to step S206 (Yes in step S205). Then, in step S206, the network connection with the imaging device 200 corresponding to the input delete button 403 is cut off and the imaging device 200 is deleted from the information processing system 10.

[0144] In step S205, if the user input is not an input to the delete button 403, the process proceeds to step S207, and the information processing device 100 performs processing in accordance with the user input other than adding or deleting a device.

[0145] FIG. 17 shows an example of a user interface using the illumination of a tally lamp 220 or the like that is normally provided in the image capture device 200. The state of calibration can be visually indicated using the tally lamp 220 or the like. For example, as shown in FIG. 17A, illumination of a specific color (e.g., green) indicates that calibration has been completed, blinking of the specific color indicates that calibration is in progress, and illumination of a color other than the specific color (e.g., red) indicates that some abnormal state has occurred. Note that the above-described illumination / blinking method is merely an example, and illumination / blinking methods are not limited to those. Furthermore, the illumination is not limited to the tally lamp 220, and any light that is provided in the image capture device 200 and that is easily visible to the user may be used.

[0146] The processing of the present technology is performed as described above. According to the present technology, the information processing device 100 calculates an optimized color conversion coefficient for each imaging device 200, and each imaging device 200 performs color gamut conversion based on the color conversion coefficient, so that the user can capture images using multiple imaging devices 200 with colors matched.

[0147] A standard color gamut (independent of Look) common to each imaging device 200 is defined, and the colors of the slave imaging devices 200 are matched to the colors of the master imaging device 200 there, and then common Look processing enables color matching using any Look among the multiple imaging devices 200. Therefore, color matching and Look processing can be performed separately.

[0148] Furthermore, calibration can be performed simply by holding a chart over the multiple imaging devices 200 used for shooting. If all imaging devices 200 used for shooting are kept in a state where they can always detect the chart while not shooting, calibration can be performed automatically as soon as the chart is detected.

[0149] The number of colors in the chart can be any number (recognized as a rectangular shape), and the more colors the chart has, the higher the accuracy of color matching. If the shooting conditions change, such as when the lighting is changed or the weather fluctuates, recalibration can be performed to enable shooting with multiple image capture devices 200 that have been color-matched under the new shooting conditions. Since color matching is optimized only for the current shooting conditions, color matching processing can be completed in a short time, allowing shooting to begin.

[0150] Furthermore, it is possible to prevent shooting in a state where the colors are not matched by not enabling shooting operations until calibration is completed for all of the image capture devices 200 used for shooting. Also, by automatically turning off chart detection during shooting, it is possible to prevent calibration from being performed during shooting.

[0151] <2. Modifications> Although the embodiments of the present technology have been specifically described above, the present technology is not limited to the above-described embodiments, and various modifications based on the technical concept of the present technology are possible.

[0152] In the embodiment, the processing target is image data, but the processing target may be video data.

[0153] In the embodiment, the information processing device 100 has been described as a device separate from the imaging device 200 , but any one of the multiple imaging devices 200 may function as the information processing device 100 .

[0154] When the information processing device 100 operates on the master imaging device 200, the information processing device 100 receives a chart image from the slave imaging device 200, calculates color conversion coefficients to match the colors of the imaging device 200 on which it operates, and transmits the color conversion coefficients to the slave imaging device 200. The information processing device 100 does not need to calculate color conversion coefficients for the imaging device 200 on which it operates.

[0155] On the other hand, when the information processing device 100 operates on a slave imaging device 200, the information processing device 100 receives a chart image from the master imaging device 200 and calculates color conversion coefficients for the slave imaging device 200 on which it operates. Furthermore, when there is a slave imaging device 200 other than the imaging device 200 on which the information processing device 100 operates, the information processing device 100 also receives a chart image from that imaging device 200, calculates color conversion coefficients for that imaging device 200, and transmits them.

[0156] Alternatively, all of the imaging devices 200 may be equipped with the functions of the information processing device 100, and each imaging device 200 may calculate its own color conversion coefficients. In this case, the master imaging device 200 transmits a chart image to all of the slave imaging devices 200, and the slave imaging devices 200 receive the chart image and calculate color conversion coefficients. Because the color conversion coefficients calculated by the slave imaging devices 200 are for their own use, there is no need to transmit the color conversion coefficients to the other imaging devices 200.

[0157] The information processing device 100 may also be configured as a cloud system. The cloud is a form of computer usage that is built on the server of a cloud service provider. Essentially, all necessary processing is performed on the server side. Users store data not on their own devices but on servers on the Internet. This allows users to use services and use, edit, and upload data in a variety of environments, such as at home, at work, on the go, on a shooting location, or in an editing room. Furthermore, a cloud system can also transfer various types of data between devices connected via a network.

[0158] When the information processing device 100 is configured as a cloud system, each imaging device 200 transmits a chart image to the cloud system via a network, receives color conversion coefficients calculated by the cloud system, and sets the color conversion coefficients for its own color gamut conversion.

[0159] In the embodiment, Look data is transmitted from the information processing device 100 to a device such as the imaging device 200, and the device processes the image data using the Look data, but the cloud system may also process the image data using the Look data.

[0160] The transmission and reception of the chart image and color conversion coefficients between the information processing device 100 and the imaging device 200 is not limited to wired or wireless communication, but may be performed via a storage medium such as a USB (Universal Serial Bus) memory or an SD card.

[0161] The present technology can also be configured as follows. (1) The system comprises an imaging device operating in a master mode, at least one imaging device operating in a slave mode, and an information processing device, the imaging device in master mode and the imaging device in slave mode photograph a specific chart and transmit the generated chart image to the information processing device; The information processing system is configured such that the information processing device calculates color conversion coefficients for the imaging device in slave mode based on the chart image generated by the imaging device in master mode, and transmits the color conversion coefficients to the corresponding imaging device. (2) The information processing system according to (1), wherein the information processing device calculates the color conversion coefficients using a color difference between the chart image generated by the imaging device in master mode and the chart image generated by the imaging device in slave mode. (3) The information processing system according to (2), wherein the information processing device calculates the color conversion coefficients so that the color difference between the chart image of the imaging device in master mode and the chart image of the imaging device in slave mode is minimized. (4) The information processing system according to any one of (1) to (3), wherein the color conversion coefficients are matrix coefficients. (5) The information processing system according to any one of (1) to (4), wherein the color conversion coefficients are a 3D LUT. (6) 2. The information processing system according to claim 1, wherein the imaging device includes a color gamut conversion unit that performs color gamut conversion based on the color conversion coefficients. (7) The information processing system according to (6), wherein the imaging device includes a Log conversion unit that performs Log conversion after the color gamut conversion by the color gamut conversion unit. (8) The information processing system according to (6), wherein the imaging device includes a look processing unit that performs look processing after the color gamut conversion by the color gamut conversion unit. (9) the imaging device includes a subject detection unit and a control unit; When a chart detection result indicating that the subject detection unit has detected the chart is transmitted to the control unit, an imaging unit photographs the chart under the control of the control unit, and generates the chart image. (10) the information processing device includes an image processing unit that performs image processing on the chart image, The information processing system according to (2), wherein the image processing unit performs processing to correct the orientation of the chart image when the color difference is equal to or greater than a predetermined value. (11) when receiving the chart image from the imaging devices, the information processing device transmits to all the imaging devices a shooting operation invalidation instruction for invalidating a shooting operation from a user; The information processing system according to any one of (1) to (10), wherein the imaging device, upon receiving the photographing operation invalid instruction, puts itself into a photographing operation invalid state. (12) when the information processing device receives notifications from all of the imaging devices in slave mode that the setting of the color conversion coefficients has been completed, it transmits a shooting operation enable instruction to enable shooting operations of all of the imaging devices; The information processing system according to (11), wherein the imaging device, upon receiving the photography operation valid instruction, changes itself from the photography operation invalid state to the photography operation valid state. (13) The information processing system according to any one of (1) to (12), wherein the information processing device sets one of the plurality of imaging devices as an imaging device that operates in a master mode. (14) The information processing system according to (13), wherein the information processing device transmits a notification that the imaging device that has been determined to be in the master mode is in master mode, and transmits a notification that the imaging device is in slave mode to imaging devices other than the imaging device that has been determined to be in master mode. (15) An information processing device including a color conversion coefficient calculation unit that calculates color conversion coefficients for an external imaging device operating in a slave mode based on a chart image generated by the external imaging device operating in a master mode by photographing a specific chart. (16) An information processing method for calculating color conversion coefficients in an external imaging device operating in a slave mode based on a chart image generated by an external imaging device operating in a master mode by photographing a specific chart. (17) An information processing program that causes a computer to execute an information processing method for calculating color conversion coefficients in an external imaging device operating in a slave mode based on a chart image generated by an external imaging device operating in a master mode by photographing a specific chart. (18) It can operate in master or slave mode, A chart image generated by photographing a specific chart is transmitted to an information processing device; When operating in the slave mode, the imaging device receives color conversion coefficients calculated by the information processing device from the information processing device, and sets the color conversion coefficients for color gamut conversion. (19) It can operate in master or slave mode, A chart image generated by photographing a specific chart is transmitted to an information processing device; A control method for an imaging device, which, when operating in the slave mode, receives color conversion coefficients calculated by the information processing device from the information processing device, and sets the color conversion coefficients for color gamut conversion. (20) It can operate in master or slave mode, A chart image generated by photographing a specific chart is transmitted to an information processing device; a control program that causes a computer to execute a control method for an imaging device that, when operating in the slave mode, receives color conversion coefficients calculated by the information processing device from the information processing device and sets the color conversion coefficients for color gamut conversion; [Explanation of symbols]

[0162] 10. Information Processing System 100 Information processing device 103 Color conversion coefficient calculation unit 200 Imaging device 204 Color gamut conversion unit 205 Log conversion section 206···Look processing section

Claims

1. The system comprises an imaging device operating in a master mode, at least one imaging device operating in a slave mode, and an information processing device, the imaging device in master mode and the imaging device in slave mode photograph a specific chart and transmit the generated chart image to the information processing device; When the information processing device receives any one of the chart images, it transmits to all of the imaging devices a shooting operation invalid instruction for invalidating a shooting operation from a user, calculates color conversion coefficients for the imaging devices in slave mode based on the chart image generated by the imaging device in master mode, and transmits the color conversion coefficients to the corresponding imaging devices; The imaging device includes a color gamut conversion unit that performs color gamut conversion based on the color conversion coefficient, and when the imaging device receives the shooting operation disable instruction, places itself in a shooting operation disabled state, and sets the color conversion coefficient for color gamut conversion in the color gamut conversion unit, and then places itself in a shooting operation enabled state. Information processing system.

2. The information processing device calculates the color conversion coefficients using a color difference between the chart image generated by the imaging device in master mode and the chart image generated by the imaging device in slave mode. The information processing system according to claim 1 .

3. The information processing device calculates the color conversion coefficients so that a difference in color between the chart image of the imaging device in master mode and the chart image of the imaging device in slave mode is minimized. The information processing system according to claim 2 .

4. The color conversion coefficients are matrix coefficients. The information processing system according to claim 1 .

5. The color transformation coefficients are a 3D LUT The information processing system according to claim 1 .

6. The imaging device includes a Log conversion unit that performs Log conversion after the color gamut conversion by the color gamut conversion unit. The information processing system according to claim 1 .

7. The imaging device includes a look processing unit that performs look processing after the color gamut conversion by the color gamut conversion unit. The information processing system according to claim 1 .

8. the imaging device includes a subject detection unit and a control unit; When the subject detection unit transmits a chart detection result indicating that the subject detection unit has detected the chart to the control unit, the imaging unit captures an image of the chart under the control of the control unit, and generates the chart image. The information processing system according to claim 1 .

9. the information processing device includes an image processing unit that performs image processing on the chart image, The image processing unit corrects the orientation of the chart image when the color difference is equal to or greater than a predetermined value. The information processing system according to claim 2 .

10. when the information processing device receives notifications from all of the imaging devices in slave mode that the setting of the color conversion coefficients has been completed, it transmits a shooting operation enable instruction to enable shooting operations of all of the imaging devices; Upon receiving the photography operation valid instruction, the imaging device changes itself from the photography operation invalid state to the photography operation valid state. The information processing system according to claim 1 .

11. The information processing device sets one of the plurality of imaging devices as an imaging device that operates in master mode. The information processing system according to claim 1 .

12. The information processing device transmits a notification of the master mode to the imaging device determined to be in the master mode, and transmits a notification of the slave mode to imaging devices other than the imaging device determined to be in the master mode. The information processing system according to claim 11.

13. a color conversion coefficient calculation unit that calculates color conversion coefficients for an external imaging device that operates in a slave mode and performs color gamut conversion based on color conversion coefficients, based on a chart image generated by the external imaging device that operates in a master mode and captures an image of a specific chart; When the chart image transmitted from the imaging device is received, a shooting operation invalid instruction for invalidating the shooting operation from the user is transmitted to all the imaging devices. Information processing device.

14. calculating color conversion coefficients for an external imaging device operating in a slave mode and performing color gamut conversion based on the color conversion coefficients, based on a chart image generated by the external imaging device operating in a master mode capturing an image of a specific chart; When the chart image transmitted from the imaging device is received, a shooting operation invalid instruction for invalidating the shooting operation from the user is transmitted to all the imaging devices. Information processing methods.

15. calculating color conversion coefficients for an external imaging device operating in a slave mode and performing color gamut conversion based on the color conversion coefficients, based on a chart image generated by the external imaging device operating in a master mode capturing an image of a specific chart; When the chart image transmitted from the imaging device is received, a shooting operation invalid instruction for invalidating the shooting operation from the user is transmitted to all the imaging devices. An information processing program that causes a computer to execute an information processing method.

16. It can operate in master or slave mode, A chart image generated by photographing a specific chart is transmitted to an information processing device; When operating in the slave mode, receiving color conversion coefficients calculated by the information processing device from the information processing device, and setting the color conversion coefficients for color gamut conversion; a color gamut conversion unit that performs color gamut conversion based on the color conversion coefficients, When receiving a shooting operation invalid instruction for invalidating a shooting operation from a user, which is transmitted from the information processing device that has received the chart image, the information processing device puts itself into a shooting operation invalid state, and after setting the color conversion coefficient for color gamut conversion in the color gamut conversion unit, puts itself into a shooting operation valid state. Imaging device.

17. It can operate in master or slave mode, A chart image generated by photographing a specific chart is transmitted to an information processing device; When operating in the slave mode, receiving color conversion coefficients calculated by the information processing device from the information processing device, and setting the color conversion coefficients for color gamut conversion; a color gamut conversion unit that performs color gamut conversion based on the color conversion coefficients, When receiving a shooting operation invalid instruction for invalidating a shooting operation from a user, which is transmitted from the information processing device that has received the chart image, the information processing device puts itself into a shooting operation invalid state, and after setting the color conversion coefficient for color gamut conversion in the color gamut conversion unit, puts itself into a shooting operation valid state. A method for controlling an imaging device.

18. It can operate in master or slave mode, A chart image generated by photographing a specific chart is transmitted to an information processing device; When operating in the slave mode, receiving color conversion coefficients calculated by the information processing device from the information processing device, and setting the color conversion coefficients for color gamut conversion; a color gamut conversion unit that performs color gamut conversion based on the color conversion coefficients, When receiving a shooting operation invalid instruction for invalidating a shooting operation from a user, which is transmitted from the information processing device that has received the chart image, the information processing device puts itself into a shooting operation invalid state, and after setting the color conversion coefficient for color gamut conversion in the color gamut conversion unit, puts itself into a shooting operation valid state. A control program that causes a computer to execute a control method for an imaging device.

Images