Information processing system, information processing device, and control method

Abstract

To suppress, by a simple method, the deterioration of color reproduction of an image signal caused by the mixture of various standards. [Solution] The information processing system includes an imaging device and an information processing device. The imaging device encodes an imaging signal or a test image signal obtained by capturing an image of a subject using a first standard and outputs the encoded image signal. The information processing device includes an image processing unit that decodes the image signal output from the imaging device using a second standard and outputs the decoded image signal, a detection unit that detects a mismatch between the first standard and the second standard based on the signal level of the image signal decoded by the image processing unit, and a correction unit that, when the detection unit detects a mismatch between the first standard and the second standard, determines a correction content for correcting the mismatch between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit and performs a correction process by applying the correction content to the encoded image signal acquired from the imaging device.

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, and a control method. [Background technology]

[0002] There are a variety of standards for image signal formats, and among them there are multiple definitions of color space and range used (see, for example, Patent Document 1). If the color space and range standards of the image signal are not properly transmitted to an information processing device that handles the image signal output from a camera, the color information that the camera is supposed to send may not be properly reproduced, even if the image itself is visible. Therefore, it is basically desirable to ensure that the standards are consistent between the camera and the information processing device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-115736 Summary of the Invention [Problem to be solved by the invention]

[0004] However, digital devices derived from earlier television broadcasts often used matrix coefficients based on the color space standards used in earlier television broadcasts (for example, matrix coefficients defined by ITU-R BT.601) and limited-range settings, but as pixel counts increased and content became HD-compatible, matrix coefficients defined by ITU-R BT.709 began to be used. Meanwhile, general personal computers, which originally tended to handle still images and graphics, often used matrix coefficients defined by ITU-R BT.601, and customarily used full-range settings for the range, as specified in Exif JPEG recording. Subsequently, images with HD pixel counts and higher began to be used frequently on personal computers, resulting in a diverse mix of operating modes even before industry standards were established.

[0005] In principle, any system violations should be treated as defects and addressed. However, recent developments in imaging equipment and software have made it easier for even non-experts to use existing libraries and processors. Furthermore, errors in matrix coefficients and range settings, in particular, are visible in the image, making them difficult to detect as defects at first glance, leading to a lack of progress in addressing these issues. Furthermore, the expansion of network use, coupled with the need for relay processing and reuse, makes it difficult to identify and address standard discrepancies. As a result, the market is faced with a diverse mix of standards and practices, and accurate standard transmission between devices cannot be expected. As a result, the color reproduction of image signals can be impaired, posing a difficult problem to address.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and one of its objects is to provide an information processing system, an information processing device, and a control method that can simply prevent the color reproduction of image signals from being impaired due to the mixing of various standards. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and according to a first aspect of the present invention, there is provided an information processing system including an imaging device and an information processing device that acquires an image signal output from the imaging device and performs processing to display on a display unit a display image based on the acquired image signal, wherein the imaging device comprises an imaging section that outputs an imaging signal obtained by capturing an image of a subject, a test signal output section that outputs a preset test image signal, and an imaging processing section that encodes the imaging signal output from the imaging section or the test image signal output from the test signal output section in a first standard set in the imaging device and outputs the encoded image signal, and the information processing device encodes the image signal output from the imaging device in a second standard and outputting the decoded image signal; a detection unit that acquires an image signal after the image processing unit decodes the encoded image signal of the test image signal acquired from the imaging device and detects a mismatch between the first standard and the second standard based on a signal level of the acquired decoded image signal; and a correction unit that, when the detection unit detects a mismatch between the first standard and the second standard, determines a correction content to correct the mismatch between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit, wherein the correction unit performs a correction process by applying the correction content to the encoded image signal of the imaging signal acquired from the imaging device.

[0008] In the above information processing system, the levels of the image signal when the test image signal is encoded in the first standard and then decoded in the second standard using a combination of multiple types of the first standard and multiple types of the second standard are each preset as multiple detection patterns used by the detection unit when detecting an inconsistency between the first standard and the second standard, and the detection unit may detect an inconsistency between the first standard and the second standard by comparing the image signal after the encoded image signal of the test image signal obtained from the imaging device and the image signal after decoded by the image processing unit with the multiple detection patterns.

[0009] In the above information processing system, each of the detection patterns among the plurality of detection patterns that is a combination of an inconsistency between the first standard and the second standard is associated with and set to a respective correction content for correcting the inconsistency between the first standard and the second standard, and when the detection unit detects an inconsistency between the first standard and the second standard, the correction unit may perform the correction process by applying the correction content associated with the corresponding detection pattern among the plurality of detection patterns.

[0010] In the above information processing system, the first standard and the second standard may include a color space standard, and the correction content may include a conversion formula for converting the color space.

[0011] In the above information processing system, the first standard and the second standard may include a standard for the number of gradations, and the correction content may include a conversion formula for converting the number of gradations.

[0012] In the above information processing system, when the detection unit detects an inconsistency between the first standard and the second standard, it may send an instruction to the imaging device to output the test image signal, and when the imaging processing unit receives the instruction to output the test image signal, it may encode the test image signal output from the test signal output unit in the first standard and output the encoded image signal.

[0013] Furthermore, according to a second aspect of the present invention, an information processing device acquires an image signal output from an imaging device, the image signal being output from the imaging device; the information processing device selects either an imaging signal obtained by capturing an image of a subject or a preset test image signal, encodes the image signal in a first standard set in the imaging device, and outputs the encoded image signal; and displays an image based on the acquired image signal on a display unit. The information processing device includes: an image processing unit that decodes the image signal output from the imaging device in a second standard and outputs the decoded image signal; a detection unit that instructs the imaging device to output the test image signal, thereby acquiring an image signal after the image processing unit decodes the encoded image signal of the test image signal acquired from the imaging device, and detects a mismatch between the first standard and the second standard based on a signal level of the acquired decoded image signal; and a correction unit that, when the detection unit detects a mismatch between the first standard and the second standard, determines a correction content to correct the mismatch between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit. The correction unit performs a correction process by applying the correction content to the encoded image signal of the imaging signal acquired from the imaging device.

[0014] Furthermore, according to a third aspect of the present invention, there is provided a control method for an information processing device that acquires an image signal output from an imaging device, the image signal being selected from an imaging signal obtained by capturing an image of a subject or a preset test image signal, encoding the image signal in a first standard set in the imaging device, and outputting the encoded image signal, and that displays on a display unit an image based on the acquired image signal. The control method includes the steps of: an image processing unit decoding the image signal output from the imaging device in a second standard and outputting the decoded image signal; a detection unit instructing the imaging device to output the test image signal, thereby acquiring an image signal after the image processing unit decodes the encoded image signal of the test image signal acquired from the imaging device, and detecting a mismatch between the first standard and the second standard based on a signal level of the acquired decoded image signal; a correction unit, when the detection unit detects a mismatch between the first standard and the second standard, determining a correction content for correcting the mismatch between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit; and a correction process by the correction unit applying the correction content to the encoded image signal of the imaging signal acquired from the imaging device. [Effects of the Invention]

[0015] According to the above aspect of the present invention, it is possible to prevent the color reproduction of an image signal from being impaired due to the coexistence of various standards in a simple manner. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a configuration diagram showing an example of an information processing system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing system according to a first embodiment. [Figure 3] FIG. 1 is a block diagram showing an example of the functional configuration of an information processing system according to a first embodiment. [Figure 4] FIG. 3 is a diagram showing an example of a test image signal used to detect a mismatch in standards according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing an example of a detection pattern for inconsistency in matrix coefficients of a color space according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing another example of a detection pattern for detecting mismatch in matrix coefficients of a color space according to the first embodiment. [Figure 7] FIG. 4 is a diagram showing an example of a detection pattern for range mismatch according to the first embodiment. [Figure 8] 5A and 5B are diagrams showing examples of determination and correction of inconsistency in matrix coefficients for each detection pattern according to the first embodiment. [Figure 9] 5A and 5B are diagrams showing examples of determination and correction of range mismatch for each detection pattern according to the first embodiment. [Figure 10] 6 is a flowchart showing an example of color reproduction compensation processing on the camera side according to the first embodiment. [Figure 11] 10 is a flowchart showing an example of color reproduction compensation processing on the information processing device side according to the first embodiment. [Figure 12] FIG. 10 is a configuration diagram showing an example of an information processing system 1A according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. An information processing system according to an embodiment of the present invention includes a camera and an information processing device that acquires an image signal output from the camera and performs processing to display a display image based on the acquired image signal. The camera may be built into the information processing device, or may be an external device (peripheral device) connected to the information processing device.

[0018] First Embodiment First, a first embodiment of the present invention will be described. FIG. 1 is a configuration diagram showing an example of an information processing system 1 according to this embodiment. As shown in FIG. 1, an information processing system 1 according to this embodiment is configured as an information processing device 10 incorporating a camera 50. The information processing device 10 is an example of a computer device, such as a clamshell (notebook) PC (Personal Computer). The camera 50 is provided in the information processing device 10 as an example of an imaging device that captures an image of a subject. The information processing device 10 displays an image captured by the camera 50 on a display unit 15.

[0019] (Hardware configuration of information processing device 10) 2 is a block diagram showing an example of the hardware configuration of an information processing device 10 according to this embodiment. The illustrated information processing device 10 includes a display unit 15, a USB connector 16, a communication unit 17, a storage unit 18, an input unit 19, an EC (Embedded Controller) 20, a power supply unit 21, a battery 22, and a system processing unit 100.

[0020] The display unit 15 includes a liquid crystal display (LCD), an organic electroluminescence (EL) display, or the like. The display unit 15 displays an image based on display data under the control of the system processing unit 100. The display data includes still images, moving images, and text data acquired or generated by processing of the OS or an application running on the OS. For example, when a camera 50 and the information processing device 10 are connected as shown in FIG. 1, the display unit 15 displays an image based on an image signal acquired from the camera 50 based on processing of the application, etc.

[0021] The USB connector 16 is a connection terminal for connecting peripheral devices that use a USB (Universal Serial Bus). For example, the USB connector 16 is a terminal on the receptacle side that complies with the USB Type-C standard.

[0022] The communication unit 17 is communicably connected to other devices via a wireless or wired communication network, and transmits and receives various types of data. For example, the communication unit 17 is configured to include a wired LAN interface such as Ethernet (registered trademark) or a wireless LAN interface such as Wi-Fi (registered trademark).

[0023] The storage unit 18 includes storage media such as a hard disk drive (HDD), a solid state drive (SSD), a random access memory (RAM), a read only memory (ROM), etc. For example, the storage unit 18 stores an OS, various drivers, various services / utilities, programs such as applications, various data, etc.

[0024] The input unit 19 is an input unit that accepts user input, and is configured to include a keyboard, for example, as in the information processing device 10 shown in FIG. 1. In response to accepting a user operation on the keyboard, the input unit 19 outputs an operation signal corresponding to the user operation to the EC 20. Note that the input unit 19 may be configured to include a touch panel, a touch pad, or the like instead of or in addition to a keyboard. Furthermore, the input unit 19 may be connected to an external operation device such as a mouse or an external keyboard via a wired or wireless connection, and may accept a user operation on the connected external operation device.

[0025] The EC20 is a one-chip microcomputer that monitors and controls various devices (peripheral devices, sensors, etc.) regardless of the system state of the OS. The EC20 is configured to include a CPU (Central Processing Unit), RAM, ROM, etc. (not shown), and also includes A / D input terminals for multiple channels, D / A output terminals, a timer, and digital input / output terminals. The EC20 is connected to an input unit 19, a power supply unit 21, a system processing unit 100, etc. via these input / output terminals. The EC20 receives and transmits various signals from each of the connected units.

[0026] For example, the EC 20 acquires an operation signal output from the input unit 19 and executes processing based on the acquired operation signal. The EC 20 outputs, to the system processing unit 100, an operation signal related to processing by the system processing unit 100 among the acquired operation signals. The EC 20 also controls the power supply unit 21 in accordance with the system state of the OS, etc. For example, the EC 20 outputs, to the power supply unit 21, a control signal for controlling the supply of power in accordance with the system state, etc. The EC 20 also acquires information on the state (remaining capacity, etc.) of the battery 22 from the power supply unit 21 by communicating with the power supply unit 21.

[0027] The power supply unit 21 includes, for example, a DC / DC converter and a charge / discharge circuit for controlling charging or discharging of the battery 22. The power supply unit 21 converts DC power supplied from the battery 22 or DC power supplied from an external power supply (such as an AC adapter) (not shown) into a plurality of voltages required to operate each unit of the information processing device 10. The power supply unit 21 supplies power to each unit of the information processing device 10 under the control of the EC 20.

[0028] The battery 22 is a secondary battery for supplying power to each unit of the information processing device 10 when power is not supplied from an external power source (such as an AC adapter). When power is supplied from an external power source (such as an AC adapter), the battery 22 is charged with that power via the power supply unit 21 until it is fully charged. When power is not supplied from the external power source (such as an AC adapter), the power charged in the battery 22 is discharged and supplied to each unit of the information processing device 10 via the power supply unit 21.

[0029] The system processing unit 100 includes a CPU 101, a GPU (Graphic Processing Unit) 102, a memory controller 103, an I / O (Input-Output) controller 104, and a system memory 105. The CPU 101 and the GPU 102 may be collectively referred to as a processor.

[0030] The CPU 101 executes processes according to programs such as an OS, various drivers, various services / utilities, and applications. As an example of the OS, Windows (registered trademark) is applied. The GPU 102 is connected to the display unit 15. The GPU 102 executes image processing under the control of the CPU 101 to generate display data. The GPU 102 outputs the generated display data to the display unit 15. The CPU 101 and the GPU 102 may be integrated into one core, or the CPU 101 and the GPU 102 may be formed as individual cores with the load shared between them. The number of processors is not limited to one, and may be multiple.

[0031] The memory controller 103 controls the reading and writing of data from the system memory 105, the storage unit 18, etc., by the processing of the CPU 101 and the GPU 102. The I / O controller 104 controls the input and output of data to and from the display unit 15, the USB connector 16, the communication unit 17, the EC 20, the camera 50, and the like.

[0032] The system memory 105 is a writable memory used as a read area for programs executed by processors such as the CPU 101 and GPU 102, or as a work area for writing processing data for the programs. For example, the system memory 105 includes multiple DRAM (Dynamic Random Access Memory) chips. The programs include an OS, various drivers for controlling peripheral devices, various service / utility programs, application programs, etc.

[0033] Camera 50 outputs an image signal based on an imaging signal obtained by capturing an image of a subject. For example, camera 50 is connected to I / O controller 104 via USB. Note that camera 50 is not limited to a built-in camera connected via USB, and may be a built-in camera connected via MIPI (Mobile Industry Processor Interface), etc.

[0034] (Image signal correction processing) Next, a correction process for an image signal that is performed when an image captured by the camera 50 is displayed on the display unit 15 of the information processing device 10 will be described. Fig. 3 is a block diagram showing an example of the functional configuration of the information processing system 1 according to this embodiment. This diagram shows an example of the functional configuration for performing correction processing of an image signal in the information processing system 1. In Fig. 3, the same reference numerals are used to designate components corresponding to those in Figs. 1 and 2.

[0035] Camera 50 includes an imaging unit 51, an imaging processing unit 52, and a test signal generator 53. Imaging unit 51 includes a lens and an imaging element (not shown), and outputs an imaging signal obtained by capturing an image of a subject to imaging processing unit 52. Imaging processing unit 52 encodes the imaging signal output from imaging unit 51 in accordance with a standard set in camera 50, and outputs the encoded image signal.

[0036] The information processing device 10 includes a camera driver 110, an application processing unit 111, a detection unit 112, and a correction unit 113 as functional components realized by the system processing unit 100 executing programs such as applications and drivers. The system processing unit 100 acquires an encoded image signal output from the camera 50 via the I / O controller 104 (see FIG. 2), decodes the acquired image signal, and displays it on the display unit 15.

[0037] The camera driver 110 is a functional configuration realized by executing a program for enabling the use of the camera 50. For example, the camera driver 110 decodes an image signal acquired from the camera 50 and makes it available for use by an application.

[0038] The application processing unit 111 is a functional configuration realized by executing various application programs. For example, in processing of an application that performs processing using an image signal acquired from the camera 50, the application processing unit 111 performs image processing on a decoded image signal output from the camera driver 110. At this time, the application processing unit 111 outputs the decoded image signal output from the camera driver 110 and the image signal after image processing to the display unit 15. In other words, the image signal acquired by the system processing unit 100 from the camera 50 is sent to the display unit 15 via the camera driver 110 and the application processing unit 111, and an image based on the decoded image signal is displayed on the display unit 15.

[0039] The standard used when decoding in the information processing device 10 (system processing unit 100) depends on the hardware or software that performs the decoding within the information processing device 10. Here, an example will be described in which the camera driver 110 performs the decoding, but the application (application processing unit 111) may perform the decoding. Alternatively, the display driver that controls the display unit 15 may perform the decoding. Alternatively, the function of the camera driver 110 may be realized by hardware, and the decoding may be performed by hardware. Note that the encoding or decoding of the image signal may be performed multiple times before the image is displayed on the display unit 15. For example, the image signal may be decoded by both the camera driver 110 and the application processing unit 111.

[0040] Here, standards related to encoding and decoding of image signals will be described. Image signals of still images or videos handled by the information processing device 10 and camera 50 are converted from YUV to RGB color space or from RGB to YUV color space according to a predetermined standard (format). There are several types of standards, such as "ITU-R BT.601," the standard for previous television broadcasts (SDTV: Standard Definition Television), and "ITU-R BT.709," the standard for HDTV (High-Definition Television). These multiple types of standards differ depending on the specifications of the camera 50 and the specifications of the hardware or software of the information processing device 10, and multiple types of standards are mixed on the market.

[0041] If the standard used for encoding by camera 50 differs from the standard used for decoding by information processing device 10, the matrix coefficients for color space conversion will differ, and color reproduction may be impaired when an image captured by camera 50 is displayed on information processing device 10. For example, if an image signal encoded using matrix coefficients for color space conversion defined in "ITU-R BT.601" is decoded using the same standard, the color information will generally be converted correctly.

[0042] On the other hand, if an image signal encoded with the matrix coefficients for color space conversion defined in "ITU-R BT.601" is decoded with the matrix coefficients for color space conversion defined in a different standard, "ITU-R BT.709," the color information will not be converted correctly.Similarly, if an image signal encoded with the matrix coefficients for color space conversion defined in "ITU-R BT.709" is decoded with the matrix coefficients for color space conversion defined in a different standard, "ITU-R BT.601," the color information will not be converted correctly.

[0043] Furthermore, there are two ranges (number of gradations) for displaying 8-bit data image signals: full range, which displays 256 gradations from "0 to 255," and limited range, which displays 220 gradations limited to "16 to 235." In the full range, the range of "0 to 255" corresponds to the color space, while in the limited range, the range of "16 to 235" corresponds to the color space. Therefore, it is desirable to decode an image signal encoded in the full range in the full range, and similarly, it is desirable to decode an image signal encoded in the limited range in the limited range.

[0044] If a range mismatch occurs, such as when an image signal encoded in full range is decoded in limited range, or when an image signal encoded in limited range is decoded in full range, the RGB gradation may not be converted correctly, and as a result, the color information may not be converted correctly.

[0045] For example, if the correct standard information is attached to the image signal output from the camera 50, it will be possible to know which standard was used for encoding, but in reality, incorrect standard information may be attached, or the standard information may not be attached at all. While the ideal measure would be to promote the correct use of standards, in environments such as PCs that rely on many existing devices, it is expected that this will be difficult to realize, or that it will be realized in the near future.

[0046] Therefore, the information processing system 1 according to this embodiment does not rely on standard information, but instead detects and corrects errors in image signal conversion caused by standard mismatch. As described above, multiple types of standards coexist, but it is possible to estimate the error pattern after image signal conversion when a standard mismatch occurs. Therefore, by using an image signal (referred to as a "test image signal") of a specific test image (test image), the information processing system 1 can detect standard mismatch and determine correction details by comparing the image signal after encoding and decoding with the error pattern. For example, the information processing system 1 prepares a detection pattern including an error pattern in advance to detect a standard mismatch (mismatch) between encoding and decoding standards.

[0047] 4 to 6, a detection pattern for detecting mismatches in matrix coefficients of a color space will be described. 4A and 4B are diagrams showing an example of a test image signal used to detect a standard mismatch according to this embodiment. (A) of FIG. 4 shows an example of a test image signal used to detect a standard mismatch. Here, a 75% color bar is used as the test image. That is, an image signal of a 75% color bar is used as the test image signal.

[0048] (B) in Figure 4 is a graph showing the RGB levels (8-bit data values) of the test image signal. If the RGB levels of the image signal after encoding and decoding the test image signal are in the state shown in (B) in Figure 4, it can be determined that the encoding matrix coefficients and the decoding matrix coefficients match, i.e., that the encoding matrix coefficients and the decoding matrix coefficients are consistent. In other words, the RGB levels shown in (B) in Figure 4 can be used as a detection pattern to detect whether the encoding and decoding matrix coefficients are consistent. The ID indicating the process in which the encoding and decoding matrix coefficients are consistent is set to "M0."

[0049] FIG. 5 is a diagram showing an example of a detection pattern for matrix coefficient mismatches in color spaces according to this embodiment. (A) in FIG. 5 is a graph showing RGB levels (8-bit data values) when a test image signal (75% color bar) is encoded using "ITU-R BT.709" and decoded using "ITU-R BT.601." If the RGB levels of the image signal after encoding and decoding the test image signal are as shown in (A) in FIG. 5, it can be determined that the standards are mismatched. In other words, the RGB levels shown in (A) in FIG. 5 are a detection pattern for detecting matrix coefficient mismatches when encoding is performed using "ITU-R BT.709" and decoding is performed using "ITU-R BT.601," and the ID indicating this encoding and decoding process is "Ma1."

[0050] Fig. 5(B) is a graph showing RGB levels (8-bit data values) when the process of "Ma1" in Fig. 5(A) is performed twice. In other words, the RGB levels shown in Fig. 5(B) are a detection pattern for detecting inconsistencies in matrix coefficients when encoding using "ITU-R BT.709" and decoding using "ITU-R BT.601" are performed twice, and the ID indicating the process of performing this "Ma1" process twice is "Ma2."

[0051] FIG. 6 is a diagram showing another example of a detection pattern for detecting mismatches in matrix coefficients of color spaces according to this embodiment. (A) in FIG. 6 is a graph showing RGB levels (8-bit data values) when a test image signal (75% color bar) is encoded using "ITU-R BT.601" and decoded using "ITU-R BT.709." If the RGB levels of the image signal after encoding and decoding the test image signal are as shown in (A) in FIG. 6, it can be determined that the standards are mismatched. In other words, the RGB levels shown in (A) in FIG. 6 are a detection pattern for detecting mismatches in matrix coefficients when encoding is performed using "ITU-R BT.601" and decoding is performed using "ITU-R BT.709." The ID indicating this encoding and decoding process is "Mb1."

[0052] Fig. 6(B) is a graph showing the RGB levels (8-bit data values) when the process of "Mb1" in Fig. 6(A) is performed twice. In other words, the RGB levels shown in Fig. 6(B) are a detection pattern that detects inconsistencies in matrix coefficients when encoding using "ITU-R BT.601" and decoding using "ITU-R BT.709" are performed twice, and the ID indicating the process of performing this "Mb1" process twice is "Mb2."

[0053] Next, a detection pattern for detecting range mismatch will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of a detection pattern for range mismatch according to this embodiment. When detecting range mismatch, for example, the RGB level (8-bit data value) of white in the test image signal (75% color bar) shown in Fig. 4(A) is used.

[0054] The graph on the left side of Figure 7 shows the RGB levels (8-bit data values) of white in the test image signal (75% color bar). When the test image signal is encoded and decoded, if the RGB levels of white in the decoded image signal are in this state (the same level as the test image signal), it can be determined that the encoding range and the decoding range match, i.e., that the encoding range and the decoding range are consistent. The ID indicating this process in which the encoding and decoding ranges are consistent is "R0." In other words, the RGB levels indicated by "R0" can be used as a detection pattern to detect whether the encoding and decoding ranges are consistent.

[0055] The ID for the process of encoding in the full range and decoding in the limited range is "Ra." Here, the ID for the process of encoding in the full range and decoding in the limited range performed once is "Ra1," and the ID for the process of performing this "Ra1" process twice is "Ra2." In other words, the white RGB level shown in "Ra1" is a detection pattern for a range mismatch when full-range encoding and limited-range decoding are performed once. The white RGB level shown in "Ra2" is a detection pattern for a range mismatch when full-range encoding and limited-range decoding are performed twice.

[0056] Furthermore, the ID for the process of encoding using a limited range and decoding using a full range is "Rb." Here, the ID for the process of encoding using a full range and decoding using a limited range once is "Rb1," and the ID for the process of performing this "Rb1" process twice is "Rb2." In other words, the white RGB level shown in "Rb1" is a detection pattern for a range mismatch when limited range encoding and full range decoding are performed once. Furthermore, the white RGB level shown in "Rb2" is a detection pattern for a range mismatch when limited range encoding and full range decoding are performed twice.

[0057] Next, the determination content when detecting mismatches in matrix coefficients and mismatches in ranges and the correction content when mismatches are detected will be described.

[0058] First, the determination details and correction details for mismatch of matrix coefficients will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the determination details and correction details for mismatch of matrix coefficients for each detection pattern according to this embodiment. Fig. 8 shows the IDs ("M0", "Ma1", "Ma2", "Mb1", "Mb2") of each process described with reference to Figs. 4 to 6, each detection pattern (contents of each process), the estimated level of the image signal output by each process, and the correction details, in association with each other.

[0059] "M0" is a detection pattern of a process in which the matrix coefficients of encoding and decoding are consistent (matched), and the estimated RGB level of the output image signal at this time (i.e., the RGB level of the test image signal) "v Ii " is expressed by the following formula 1. Note that "i" indicates each color of the 75% color bar, and each color is associated with "i=1 to 7".

[0060]

number

[0061] "Ma1" is a detection pattern for the process of encoding "ITU-R BT.709" and decoding "ITU-R BT.601". The matrix coefficients of "ITU-R BT.601" are "M 601 " and the matrix coefficients of "ITU-R BT.709" are "M 709 ", the estimated RGB level of the output image signal "v Ma1i " can be expressed by the following equation 2.

[0062]

number

[0063] In addition, "f Ma1 ()" is a function that calculates the RGB levels of the output when "Ma1" processing is performed on the RGB levels of the test image signal. Here, "f Ma1 ()=(M 601 ) -1 M 709 " is defined as:

[0064] In the case of a mismatch in matrix coefficients due to the "Ma1" process, the correction content can be expressed by the following equation 3, where the RGB level of the image signal to be corrected is "v0".

[0065]

number

[0066] In other words, in the case of a mismatch in the matrix coefficients due to the "Ma1" processing, the RGB levels of the image signal to be corrected are set to "f Ma1 This can be corrected using the inverse function of ().

[0067] Next, "Ma2" is a detection pattern that performs the "Ma1" process twice, and the estimated RGB level of the output image signal is "v Ma2i " can be expressed by the following equation 4.

[0068]

number

[0069] In addition, "f Ma2 ()" is a function that calculates the RGB levels of the output when the "Ma1" process is performed twice on the RGB levels of the test image signal. Here, "f Ma2 ()=((M 601 ) -1 M 709 ) 2 " is defined as:

[0070] In the case of a mismatch in matrix coefficients due to this "Ma2" processing, the correction content can be expressed by the following equation 5, where the RGB level of the image signal to be corrected is "v0".

[0071]

number

[0072] In other words, in the case of a mismatch in the matrix coefficients due to the "Ma2" processing, the RGB levels of the image signal to be corrected are set to "f Ma2 This can be corrected using the inverse function of ().

[0073] Next, "Mb1" is the detection pattern for the encoding process "ITU-R BT.601" and the decoding process "ITU-R BT.709", and the estimated RGB level of the output image signal "v Mb1i " can be expressed by the following equation 6.

[0074]

number

[0075] In addition, "f Mb1 ()" is a function that calculates the RGB levels of the output when "Mb1" processing is performed on the RGB levels of the test image signal. Here, "f Mb1 ()=(M 709 ) -1 M 601 " is defined as:

[0076] In the case of a mismatch in matrix coefficients due to the processing of "Mb1", the correction content can be expressed by the following equation 7, where the RGB level of the image signal to be corrected is "v0".

[0077]

number

[0078] In other words, in the case of a mismatch in the matrix coefficients due to the processing of "Mb1", the RGB levels of the image signal to be corrected are set to "f Mb1 This can be corrected using the inverse function of ().

[0079] Next, "Mb2" is a detection pattern that performs the "Mb1" process twice, and the estimated RGB level of the output image signal is "v Mb2i " can be expressed by the following equation 8.

[0080]

number

[0081] In addition, "fMb2 ()" is a function that calculates the RGB levels of the output when the "Mb1" process is performed twice on the RGB levels of the test image signal. Here, "f Mb2 ()=((M 709 ) -1 M 601 ) 2 " is defined as:

[0082] In the case of a mismatch in matrix coefficients due to the processing of "Mb2", the correction content can be expressed by the following equation 9, where the RGB level of the image signal to be corrected is "v0".

[0083]

number

[0084] In other words, in the case of a mismatch in the matrix coefficients due to the "Mb2" processing, the RGB levels of the image signal to be corrected are set to "f Mb2 This can be corrected using the inverse function of ().

[0085] Next, the determination details and correction details of range mismatch will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the determination details and correction details of range mismatch for each detection pattern according to this embodiment. Fig. 9 shows the IDs of each process ("R0", "Ra1", "Ra2", "Rb1", "Rb2") described with reference to Fig. 7, each detection pattern (contents of each process), the estimated level of the image signal output by each process, and the correction details, all associated with each other.

[0086] "R0" is a detection pattern of a process in which the encoding and decoding ranges are consistent (matched), and the estimated RGB level of white in the output image signal at this time (i.e., the RGB level of white in the test image signal) "N Iw " can be expressed by the following equation 10.

[0087]

number

[0088] "Ra1" is a detection pattern where encoding is full range and decoding is limited range processing, and the full range gradation is "0 to 255" and the limited range gradation is "16 to 235", so the estimated RGB level of the output image signal is "N Ra1 " can be expressed by the following equation 11.

[0089]

number

[0090] In addition, "f Ra1 ()" is a function for calculating the RGB levels of the output when "Ra1" processing is performed on the RGB levels of the test image signal, and is defined as in Equation 11 above.

[0091] In the case of a range mismatch caused by this "Ra1" processing, the correction content can be expressed by the following equation 12, where the RGB level of the image signal to be corrected is "N0".

[0092]

number

[0093] In other words, in the case of a range mismatch caused by "Ra1" processing, the RGB levels of the image signal to be corrected are set to "f Ra1 This can be corrected using the inverse function of ().

[0094] Next, "Ra2" is a detection pattern that performs the "Ra1" process twice, and the estimated RGB level of the output image signal is "N Ra2 " can be expressed by the following equation 13.

[0095]

number

[0096] In addition, "fRa2 ()" is a function that calculates the RGB levels of the output when the "Ra1" process is performed twice on the RGB levels of the test image signal, and is defined as in Equation 13 above.

[0097] In the case of a range mismatch caused by this "Ra2" processing, the correction content can be expressed by the following equation 14, where the RGB level of the image signal to be corrected is "N0".

[0098]

number

[0099] In other words, in the case of a range mismatch caused by "Ra2" processing, the RGB levels of the image signal to be corrected are set to "f Ra2 This can be corrected using the inverse function of ().

[0100] Next, "Rb1" is a detection pattern for limited-range encoding and full-range decoding, and the estimated RGB level of the output image signal is "N Rb1 " can be expressed by the following equation 15.

[0101]

number

[0102] In addition, "f Rb1 ()" is a function for calculating the RGB levels of the output when "Rb1" processing is performed on the RGB levels of the test image signal, and is defined as in Equation 15 above.

[0103] In the case of a range mismatch caused by this "Rb1" processing, the correction content can be expressed by the following equation 16, where the RGB level of the image signal to be corrected is "N0".

[0104]

number

[0105] In other words, in the case of a range mismatch due to "Rb1" processing, the RGB levels of the image signal to be corrected are set to "f Rb1 This can be corrected using the inverse function of ().

[0106] Next, "Rb2" is a detection pattern that performs the "Rb1" process twice, and the estimated RGB level of the output image signal is "N Rb2 " can be expressed by the following equation 17.

[0107]

number

[0108] In addition, "f Rb2 ()" is a function that calculates the RGB levels of the output when the "Rb1" process is performed twice on the RGB levels of the test image signal, and is defined as in Equation 17 above.

[0109] In the case of a range mismatch caused by the "Rb2" processing, the correction content can be expressed by the following equation 18, where the RGB level of the image signal to be corrected is "N0".

[0110]

number

[0111] In other words, in the case of a range mismatch caused by "Rb2" processing, the RGB levels of the image signal to be corrected are set to "f Rb2 This can be corrected using the inverse function of ().

[0112] Next, returning to Fig. 3, a configuration for detecting and correcting inconsistencies in the standards (matrix coefficients and ranges of the color space) described with reference to Figs. 4 to 9 will be described. This processing is intended to prevent the color reproduction of the image signal from being impaired, and will therefore be referred to as "color reproduction compensation processing" in the following description.

[0113] The test signal generator 53 included in the camera 50 is configured to output a test image signal used in color reproduction compensation processing. For example, the test signal generator 53 outputs an image signal of a 75% color bar shown in FIG. 4 as the test image signal.

[0114] The imaging processing unit 52 can encode not only the imaging signal output from the imaging unit 51 but also the test image signal output from the test signal generator 53 in accordance with the standard set in the camera 50 and output the encoded image signal. In normal processing, the imaging processing unit 52 encodes and outputs the imaging signal, but when an instruction to output a test image signal is received from the information processing device 10, the imaging processing unit 52 switches to processing to encode and output the test image signal.

[0115] For example, the test signal generator 53 outputs a test image signal in response to an instruction from the image capture processing unit 52 only when the image capture processing unit 52 performs processing to encode and output a test image signal.

[0116] Furthermore, the detection unit 112 and correction unit 113 included in the information processing device 10 are the main components that perform color reproduction compensation processing on the information processing device 10 side. The detection unit 112 acquires an image signal after decoding of an image signal after encoding of a test image signal acquired from the camera 50, and detects a mismatch between the encoding standard and the decoding standard based on the signal level of the acquired decoded image signal. For example, when the detection unit 112 detects a mismatch of standards, it instructs the correction unit 113 to perform correction.

[0117] When the detection unit 112 detects a mismatch between the encoding standard and the decoding standard, the correction unit 113 determines the correction content for correcting the mismatch between the encoding standard and the decoding standard based on the signal level of the decoded image signal acquired by the detection unit 112, and performs correction processing by applying the correction content to the image signal acquired from the camera 50. The processing performed by the detection unit 112 and the correction unit 113 will be described in detail below.

[0118] First, when performing color reproduction compensation processing, the detection unit 112 transmits an output instruction for a test image signal to the camera 50. For example, the detection unit 112 transmits an output instruction for a test image signal to the camera 50 when a user launches a specific application, when a user issues an instruction via a specific UI, when the application environment is reset or updated, or when the application is launched for the first time thereafter. Then, the detection unit 112 performs processing to detect a mismatch between the encoding and decoding standards of the image signal.

[0119] Specifically, the detection unit 112 acquires the decoded image signal from the encoded image signal of the test image signal acquired from the camera 50, and detects a mismatch between the encoding standard and the decoding standard based on the signal level of the acquired decoded image signal.

[0120] For example, when acquiring the decoded image signal, the detection unit 112 captures the display screen of the display unit 15 on which an image based on the decoded image signal is displayed, and detects and cuts out the test image portion from the captured image.

[0121] The detection unit 112 also detects mismatches between the encoding standard and the decoding standard by comparing the acquired decoded image signal with a plurality of detection patterns. The plurality of detection patterns are detection patterns used to detect mismatches between the matrix coefficients and ranges of the color spaces described with reference to FIGS.

[0122] For example, in detecting mismatches in matrix coefficients, as shown in Figure 8, the levels of the image signal (estimated levels of the output image signal) when a test image signal is encoded and then decoded using combinations of encoding matrix coefficients and decoding matrix coefficients (ID: "M0", "Ma1", "Ma2", "Mb1", "Mb2") are each preset as multiple detection patterns to be used when detecting mismatches in matrix coefficients.

[0123] Then, the detection unit 112 compares the RGB levels of the decoded image signal with multiple detection patterns (estimated levels of the output image signal), determines that the encoding matrix coefficients and decoding matrix coefficients of the closest (smallest difference) detection pattern are the current standard, and determines whether the two matrix coefficients are consistent or inconsistent.

[0124] For example, if the detection pattern of "M0" is closest (the difference is smallest), the detection unit 112 determines that the encoding matrix coefficients and the decoding matrix coefficients are consistent. Also, if the detection pattern of "Ma1", "Ma2", "Mb1", or "Mb2" is closest (the difference is smallest), the detection unit 112 determines that the encoding matrix coefficients and the decoding matrix coefficients are inconsistent.

[0125] 8, among the plurality of detection patterns, a detection pattern (ID: "Ma1", "Ma2", "Mb1", "Mb2") in which the encoding matrix coefficients and the decoding matrix coefficients are inconsistent is associated with a correction content for correcting the inconsistency and set to each of them. When the detection unit 112 detects a mismatch between the encoding matrix coefficients and the decoding matrix coefficients, the correction unit 113 performs a correction process by applying the correction content associated with the corresponding detection pattern among the plurality of detection patterns.

[0126] On the other hand, for detecting mismatches between ranges (full range and limited range), as shown in Figure 9, the levels of the image signal (estimated levels of the output image signal) when the test image signal is encoded and then decoded using combinations of encoding range and decoding range (ID: R0, Ra1, Ra2, Rb1, Rb2) are each preset as multiple detection patterns to be used when detecting mismatches between ranges.

[0127] Then, the detection unit 112 compares the RGB levels of the decoded image signal with multiple detection patterns (estimated levels of the output image signal), determines that the encoding range and decoding range of the closest (smallest difference) detection pattern are the current standard, and determines whether the two ranges are consistent or inconsistent.

[0128] For example, if the detection pattern of "R0" is closest (the difference is smallest), the detection unit 112 determines that the encoding range and the decoding range are consistent. Also, if the detection pattern of "Ra1", "Ra2", "Rb1", or "Rb2" is closest (the difference is smallest), the detection unit 112 determines that the encoding range and the decoding range are inconsistent.

[0129] 9, among the plurality of detection patterns, a detection pattern (ID: "Ra1", "Ra2", "Rb1", "Rb2") in which the encoding range and the decoding range are inconsistent is associated with a respective correction content for correcting the inconsistency and set. When the detection unit 112 detects a mismatch between the encoding range and the decoding range, the correction unit 113 performs a correction process by applying the correction content associated with the corresponding detection pattern among the plurality of detection patterns.

[0130] (Color reproduction compensation processing operation) Next, the operation of the color reproduction compensation process executed by the information processing system 1 will be described. First, the operation of the color reproduction compensation process on the camera 50 side will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the color reproduction compensation process on the camera side according to this embodiment.

[0131] (Step S101) In the camera 50, the imaging processing unit 52 determines whether or not an instruction to output a test image signal has been received from the information processing device 10. If the imaging processing unit 52 determines that an instruction to output a test image signal has been received from the information processing device 10 (YES), the imaging processing unit 52 proceeds to processing of step S103. On the other hand, if the imaging processing unit 52 determines that an instruction to output a test image signal has not been received from the information processing device 10 (NO), the imaging processing unit 52 proceeds to processing of step S105.

[0132] (Step S103) The imaging processing unit 52 encodes the test image signal (75% color bar) output from the test signal generator 53 and outputs the encoded image signal.

[0133] (Step S105) The imaging processing unit 52 outputs an image signal obtained by encoding the imaging signal captured by the imaging unit 51.

[0134] Next, the operation of the color reproduction compensation process on the information processing device 10 side will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the color reproduction compensation process on the information processing device side according to this embodiment.

[0135] (Step S201) In the information processing device 10, the detection unit 112 transmits an output instruction for a test image signal to the camera 50 when performing color reproduction compensation processing. Then, the process proceeds to step S203.

[0136] (Step S203) Camera driver 110 acquires from camera 50 an image signal in which the test image signal is encoded, and decodes the acquired image signal to make it usable by an application. Application processing unit 111 outputs the decoded image signal output from camera driver 110 to display unit 15, causing it to be displayed on display unit 15. Then, the process proceeds to step S205.

[0137] (Step S205) The detection unit 112 acquires the decoded image signal. For example, the detection unit 112 captures the display screen of the display unit 15.

[0138] (Step S207) The detection unit 112 determines whether or not a test image (75% color bar) has been detected from the screen-captured image in step S205. If the detection unit 112 determines that a test image has not been detected from the screen-captured image (step S207: NO), the process returns to step S205. On the other hand, if the detection unit 112 determines that a test image has been detected from the screen-captured image (step S207: YES), the process proceeds to step S209.

[0139] (Step S209) The detection unit 112 determines whether the encoding and decoding standards are consistent or inconsistent. For example, when detecting a mismatch in matrix coefficients, the detection unit 112 compares the RGB levels of the decoded image signal with multiple detection patterns (estimated levels of the output image signal) shown in FIG. 8, determines that the encoding matrix coefficients and decoding matrix coefficients of the closest (smallest difference) detection pattern are in accordance with the current standard, and determines whether the two matrix coefficients are consistent or inconsistent. When detecting a mismatch in ranges, the detection unit 112 compares the RGB levels of the decoded image signal with multiple detection patterns (estimated levels of the output image signal) shown in FIG. 9, determines that the encoding ranges and decoding ranges of the closest (smallest difference) detection pattern are in accordance with the current standard, and determines whether the two ranges are consistent or inconsistent. Then, the process proceeds to step S211.

[0140] (Step S211) The detection unit 112 determines whether or not a mismatch between the encoding and decoding standards has been detected. If the detection unit 112 determines that the encoding and decoding standards are consistent, that is, if no mismatch between the standards has been detected (step S211: NO), the detection unit 112 terminates the process because no correction is necessary. On the other hand, if the detection unit 112 determines that a mismatch between the standards has been detected (step S211: YES), the process proceeds to step S213.

[0141] (Step S213) The detection unit 112 sends the ID of the detected pattern determined to be inconsistent to the correction unit 113, and instructs the correction unit 113 to perform correction. Then, the process proceeds to step S215.

[0142] (Step S215) When the correction unit 113 receives a correction instruction from the detection unit 112, it determines the correction content associated with the ID of the detection pattern determined to be inconsistent as the correction content to be applied to the correction of the image signal, and then ends the process.

[0143] When the detection unit 112 detects a mismatch in standards and determines the correction content, the correction unit 113 performs correction processing by applying the determined correction content to the image signal based on the imaging signal captured by the camera 50 in the subsequent processing.

[0144] For example, the correction unit 113 acquires a decoded image signal output from the camera driver 110, performs correction processing on the acquired image signal, and then passes the signal to the application processing unit 111. As an example, the correction unit 113 may acquire a decoded image signal and perform correction processing using a device MFT supported as an extended function of Windows (registered trademark).

[0145] As described above, the information processing system 1 according to this embodiment includes a camera 50 (an example of an imaging device) and an information processing device 10 that acquires an image signal output from the camera 50 and performs processing to display an image based on the acquired image signal on the display unit 15. Camera 50 includes an imaging unit 51, an imaging processing unit 52, and a test signal generator 53 (an example of a test signal output unit). Imaging unit 51 outputs an imaging signal obtained by capturing an image of a subject. Test signal generator 53 outputs a preset test image signal. Imaging processing unit 52 encodes the imaging signal output from imaging unit 51 or the test image signal output from test signal output unit 53 in accordance with a first standard set in camera 50, and outputs the encoded image signal. The information processing device 10 includes a camera driver 110 (an example of an image processing unit), a detection unit 112, and a correction unit 113. The camera driver 110 decodes an image signal output from the camera 50 using a second standard and outputs the decoded image signal. The detection unit 112 acquires an image signal obtained by the camera driver 110 decoding an encoded image signal of a test image signal acquired from the camera 50, and detects a mismatch between the first standard (encoding standard) and the second standard (decoding standard) based on the RGB levels (signal levels) of the acquired decoded image signal. When the detection unit 112 detects a mismatch between the first standard and the second standard, the correction unit 113 determines a correction content for correcting the mismatch between the first standard and the second standard based on the RGB levels of the decoded image signal acquired by the detection unit 112. The correction unit 113 also performs a correction process applying the determined correction content to the encoded image signal of the imaging signal acquired from the camera 50.

[0146] As a result, the information processing system 1 detects and corrects inconsistencies between the encoding standard and the decoding standard based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information, thereby providing a simple method for preventing the color reproduction of the image signal from being impaired due to the mixing of various standards.

[0147] For example, the levels of an image signal obtained when a test image signal is encoded in the first standard using a combination of a plurality of first standards (encoding standards) and a plurality of second standards (decoding standards) and then decoded in the second standard are preset as a plurality of detection patterns used by the detection unit 112 when detecting a mismatch between the first standard and the second standard (see FIGS. 8 and 9).The detection unit 112 then detects a mismatch between the first standard and the second standard by comparing the image signal obtained from the camera 50 after encoding the test image signal and then decoded by the camera driver 110 with the plurality of detection patterns.

[0148] This allows the information processing system 1 to detect standard mismatch without relying on standard information by comparing the RGB levels after decoding the image signal encoded by the camera 50 with the detection pattern when the standard mismatch occurs.

[0149] Furthermore, for example, for each of the detection patterns in which the first standard (encoding standard) and the second standard (decoding standard) are inconsistent, correction details for correcting the inconsistency between the first standard and the second standard are associated and set (see FIGS. 8 and 9). When the detection unit 112 detects an inconsistency between the first standard and the second standard, the correction unit 113 performs correction processing by applying the correction details associated with the corresponding detection pattern from among the multiple detection patterns.

[0150] As a result, by presetting the correction content for each detection pattern when there is a standard mismatch, the information processing system 1 can perform appropriate correction corresponding to the detected mismatch detection pattern when a standard mismatch is detected.

[0151] For example, the first standard (encoding standard) and the second standard (decoding standard) include a color space standard, and the correction content includes a conversion formula for converting the color space (see FIG. 8).

[0152] This allows the information processing system 1 to detect and correct mismatches between the matrix coefficients of the encoding color space and the matrix coefficients of the decoding color space based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information.

[0153] Furthermore, for example, the first standard (encoding standard) and the second standard (decoding standard) include a range (number of gradations) standard, and the correction content includes a conversion formula for converting the range (see FIG. 9).

[0154] This allows the information processing system 1 to detect and correct mismatches between the encoding range and the decoding range based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information.

[0155] Furthermore, when detecting a mismatch between the first standard (encoding standard) and the second standard (decoding standard), detection unit 112 transmits an instruction to output a test image signal to camera 50. Upon receiving the instruction to output a test image signal, imaging processing unit 52 of camera 50 encodes the test image signal output from test signal generator 53 in accordance with the first standard and outputs the encoded image signal.

[0156] This allows the information processing system 1 to switch from the imaging signal to the test image signal and output it from the camera 50 when performing color reproduction compensation processing to detect and correct mismatches in standards.

[0157] Furthermore, the information processing device 10 according to this embodiment acquires an image signal output from a camera 50 (an example of an imaging device) and performs processing to display an image based on the acquired image signal on the display unit 15. The camera 50 selects either an image signal obtained by capturing an image of a subject or a preset test image signal, encodes the selected image signal in a first standard set in the camera 50, and outputs the encoded image signal. The information processing device 10 also includes a camera driver 110 (an example of an image processing unit), a detection unit 112, and a correction unit 113. The camera driver 110 decodes the image signal output from the camera 50 in a second standard and outputs the decoded image signal. The detection unit 112 instructs the camera 50 to output a test image signal to acquire an image signal obtained by decoding the encoded image signal of the test image signal acquired from the camera 50 by the camera driver 110, and detects a mismatch between the first standard (encoding standard) and the second standard (decoding standard) based on the RGB levels (signal levels) of the acquired decoded image signal. When the detection unit 112 detects a mismatch between the first standard and the second standard, the correction unit 113 determines the correction content for correcting the mismatch between the first standard and the second standard based on the RGB levels of the decoded image signal acquired by the detection unit 112. Furthermore, the correction unit 113 performs a correction process applying the determined correction content to the encoded image signal of the imaging signal acquired from the camera 50.

[0158] As a result, the information processing device 10 detects and corrects inconsistencies between the encoding standard and the decoding standard based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information, thereby providing a simple method for preventing the color reproduction of the image signal from being impaired due to the mixture of various standards.

[0159] Furthermore, the control method in information processing device 10 according to this embodiment is a control method in information processing device 10 that acquires an image signal output from camera 50 (an example of an imaging device) and performs processing to display a display image based on the acquired image signal on display unit 15. Camera 50 selects either an image signal capturing an image of a subject or a preset test image signal, encodes it in accordance with a first standard set in camera 50, and outputs the encoded image signal. The control method in the information processing device 10 includes the steps of: a camera driver 110 decoding the image signal output from the camera 50 in accordance with the second standard and outputting the decoded image signal; a detection unit 112 acquiring an image signal after the camera driver 110 has decoded the encoded image signal of the test image signal acquired from the camera 50 by instructing the camera 50 to output a test image signal, and detecting a mismatch between the first standard (encoding standard) and the second standard (decoding standard) based on the RGB levels (signal levels) of the acquired decoded image signal; a correction unit 113 determining, if the detection unit 112 detects a mismatch between the first standard and the second standard, a correction content for correcting the mismatch between the first standard and the second standard based on the RGB levels of the decoded image signal acquired by the detection unit 112; and a correction process by the correction unit 113 applying the determined correction content to the encoded image signal of the imaging signal acquired from the camera 50.

[0160] As a result, the control method in the information processing device 10 detects and corrects inconsistencies between the encoding standard and the decoding standard based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information, thereby providing a simple method to prevent the color reproduction of the image signal from being impaired due to the mixture of various standards.

[0161] <Second embodiment> Next, a second embodiment of the present invention will be described. In the first embodiment, a configuration in which a camera is built into an information processing device has been described, but in this embodiment, a configuration in which a camera is an external device (peripheral device) connected to an information processing device will be described.

[0162] Fig. 12 is a configuration diagram showing an example of an information processing system 1A according to this embodiment. As shown in Fig. 12, the information processing system 1A includes an information processing device 10A and a camera 50A. The basic hardware configuration of the information processing device 10A is the same as the hardware configuration of the information processing device 10 shown in Fig. 2 except that the camera 50 is removed. Note that the information processing device 10A may include the camera 50 and may also be connected to the camera 50A.

[0163] The information processing device 10A is connected to the camera 50A by wire or wirelessly, and can acquire images captured by the camera 50A and display them on the display unit 15. For example, the information processing device 10A and the camera 50A are connected by a USB cable 5 that complies with the USB Type-C standard. For example, the USB cable 5 used to connect to the camera 50A is connected to a USB connector 16 (see FIG. 2).

[0164] In this way, the color reproduction compensation process described in the first embodiment can also be applied to the information processing system 1A in which the camera 50A is connected to the information processing device 10A as an external camera.

[0165] Although the example in which the camera 50A and the information processing device 10A are connected by the USB cable 5 has been shown, a connection method other than USB may also be used.

[0166] The embodiments of the present invention have been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope of the gist of the present invention.

[0167] In the above embodiment, an example has been described in which a 75% color bar is used as the test image signal, but this is not limiting. For example, an image including at least one of the colors of a 75% color bar may be used as the test image signal.

[0168] In the above embodiment, an example has been described in which the image signal acquired from the camera 50 is decoded by the camera driver 110, but the location where the decoding is performed is not limited to the camera driver 110. For example, the application processing unit 111 may perform the decoding using a standard set in the application. Alternatively, a display driver (not shown) that controls the display unit 15 may perform the decoding.

[0169] In the above embodiment, the method of using a screen capture of the display unit 15 as a method for the detection unit 112 to acquire a decoded image signal in order to detect a standard mismatch has been described as an example, but the present invention is not limited to this. For example, when decoding is performed by the camera driver 110 as in the above embodiment, the detection unit 112 may acquire a decoded image signal output from the camera driver 110 and detect a standard mismatch.

[0170] Furthermore, in the above embodiment, two standards for encoding and decoding image signals have been described as examples: "ITU-R BT.601," the standard for previous television broadcasting (SDTV), and "ITU-R BT.709," the standard for HDTV. However, other standards may also be used. Even when three or more standards are mixed, the number of combinations of encoding and decoding standards increases, but the above embodiment can be applied by setting a detection pattern for each. Examples of other standards may include, for example, "ITU-R BT.2020," the standard for 4K / 8K UHD (Ultra High Definition) broadcasting.

[0171] The information processing device 10 described above also includes an internal computer system. A program for implementing the functions of each component of the information processing device 10 may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform processing in each component of the information processing device 10. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" may also include multiple computer devices connected via a network, including the Internet, a WAN, a LAN, a dedicated line, or other communication lines. The term "computer-readable recording medium" refers to portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into a computer system. The recording medium storing the program may also be a non-transitory recording medium such as a CD-ROM.

[0172] The recording medium also includes internal or external recording media accessible from a distribution server for distributing the program. The program may be divided into multiple parts, downloaded at different times, and then combined by each component of the information processing device 10, or each divided program may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be a medium that realizes part of the above-described functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-described functions in combination with a program already stored in the computer system.

[0173] Furthermore, some or all of the functions of the information processing device 10 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually implemented as a processor, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be implemented using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology, an integrated circuit based on that technology may be used.

[0174] Furthermore, in the above-described embodiment, an example has been described in which the information processing device 10 is a clamshell (notebook) PC, but it may also be a desktop or tablet PC. [Explanation of symbols]

[0175] 1 Information processing system, 5 USB cable, 10 Information processing device, 15 Display unit, 16 USB connector, 17 Communication unit, 18 Memory unit, 19 Input unit, 20 EC, 21 Power supply unit, 22 Battery, 50 Camera, 51 Imaging unit, 52 Imaging processing unit, 53 Test signal generator, 100 System processing unit, 101 CPU, 102 GPU, 103 Memory controller, 104 I / O controller, 105 System memory, 110 Camera driver, 111 Application processing unit, 112 Detection unit, 113 Correction unit

Claims

1. An information processing system including an imaging device and an information processing device that acquires an image signal output from the imaging device and displays a display image based on the acquired image signal on a display unit, The imaging device is an imaging unit that outputs an imaging signal obtained by imaging a subject; a test signal output unit that outputs a preset test image signal; an imaging processing unit that encodes the imaging signal output from the imaging unit or the test image signal output from the test signal output unit in accordance with a first standard set in the imaging device and outputs the encoded image signal; Equipped with The information processing device includes: an image processing unit that decodes the image signal output from the imaging device in accordance with a second standard and outputs the decoded image signal; a detection unit that acquires an image signal obtained by decoding the encoded image signal of the test image signal acquired from the imaging device by the image processing unit, and detects a mismatch between the first standard and the second standard based on a signal level of the acquired decoded image signal; a correction unit that, when the detection unit detects a mismatch between the first standard and the second standard, determines a correction content for correcting the mismatch between the first standard and the second standard based on a signal level of the decoded image signal acquired by the detection unit; and Equipped with The correction unit a correction process in which the correction content is applied to an image signal obtained by encoding the imaging signal from the imaging device; Information processing system.

2. levels of an image signal when the test image signal is encoded in the first standard and then decoded in the second standard using a combination of a plurality of types of the first standard and a plurality of types of the second standard are preset as a plurality of detection patterns used when the detection unit detects a mismatch between the first standard and the second standard, The detection unit detecting a mismatch between the first standard and the second standard by comparing an image signal obtained by decoding the encoded image signal of the test image signal acquired from the imaging device by the image processing unit with the plurality of detection patterns; The information processing system according to claim 1 .

3. correction content for correcting the mismatch between the first standard and the second standard is associated with and set to each of the detection patterns of a combination of the first standard and the second standard among the plurality of detection patterns, The correction unit When the detection unit detects a mismatch between the first standard and the second standard, the correction process is performed by applying a correction content associated with a corresponding detection pattern among the plurality of detection patterns. The information processing system according to claim 2 .

4. the first standard and the second standard include a color space standard, The correction content includes a conversion formula for converting a color space. The information processing system according to claim 1 .

5. the first standard and the second standard include a standard for the number of gradations, The correction content includes a conversion formula for converting the number of gradations. The information processing system according to claim 1 .

6. The detection unit When detecting a mismatch between the first standard and the second standard, an instruction to output the test image signal is sent to the imaging device; The imaging processing unit When receiving an instruction to output the test image signal, the test image signal output from the test signal output unit is encoded in accordance with the first standard, and the encoded image signal is output. The information processing system according to claim 1 .

7. An information processing device that acquires an image signal output from an imaging device that selects either an imaging signal obtained by imaging a subject or a preset test image signal, encodes the image signal in accordance with a first standard set in the imaging device, and outputs the encoded image signal, and performs processing to display an image based on the acquired image signal on a display unit, an image processing unit that decodes the image signal output from the imaging device in accordance with a second standard and outputs the decoded image signal; a detection unit that acquires an image signal after the image processing unit decodes the encoded image signal of the test image signal acquired from the imaging device by instructing the imaging device to output the test image signal, and detects a mismatch between the first standard and the second standard based on a signal level of the acquired decoded image signal; a correction unit that, when the detection unit detects a mismatch between the first standard and the second standard, determines a correction content for correcting the mismatch between the first standard and the second standard based on a signal level of the decoded image signal acquired by the detection unit; and Equipped with The correction unit a correction process in which the correction content is applied to an image signal obtained by encoding the imaging signal from the imaging device; Information processing device.

8. 1. A control method for an information processing device, comprising: acquiring an image signal output from an imaging device, the image signal being selected from an imaging signal obtained by imaging a subject or a preset test image signal, encoding the selected image signal in accordance with a first standard set in the imaging device, and outputting the encoded image signal; and displaying, on a display unit, a display image based on the acquired image signal, an image processing unit decoding the image signal output from the imaging device in accordance with a second standard and outputting the decoded image signal; a detection unit instructing the imaging device to output the test image signal to acquire an image signal obtained by the imaging device after encoding the test image signal and decoding the image signal by the image processing unit, and detecting a mismatch between the first standard and the second standard based on a signal level of the acquired decoded image signal; a correction unit, when the detection unit detects a mismatch between the first standard and the second standard, determining a correction content for correcting the mismatch between the first standard and the second standard based on a signal level of the decoded image signal acquired by the detection unit; a step of performing a correction process by applying the correction content to an image signal obtained by encoding the imaging signal from the imaging device by the correction unit; A control method comprising:

Images