Information processing system, information processing device, and control method

Abstract

To suppress, in a simple way, the degradation of color reproduction in image signals caused by the coexistence of various standards. [Solution] The information processing system comprises an imaging device and an information processing device. The imaging device encodes an imaging signal or test image signal of an image taken of a subject according to a first standard and outputs an encoded image signal. The information processing device comprises an image processing unit that decodes the image signal output from the imaging device according to a second standard and outputs an encoded image signal, a detection unit that detects inconsistencies between the first standard and the second standard based on the signal level of the image signal decoded by the image processing unit, and a correction unit that, if an inconsistency between the first standard and the second standard is detected by the detection unit, determines a correction content to correct the inconsistency between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit, and performs a correction process that applies the correction content to the encoded image signal of the imaging signal acquired from the imaging device.

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, and a control method. [Background technology]

[0002] There are various standards for image signal formats, and within these standards, there are multiple definitions of color spaces and ranges (see, for example, Patent Document 1). In an information processing device that handles image signals output from a camera, if the standards for the image signal's color space and range are not correctly communicated, the image itself may be visible, but the color information that the camera should be sending may not be reproduced correctly. Therefore, it is desirable that the standards are consistent between the camera and the information processing device. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2013-115736 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, digital devices derived from previous television broadcasting often used matrix coefficients (e.g., matrix coefficients defined by ITU-R BT.601) and limited range settings based on the color space standards used in previous television broadcasting. As the number of pixels increased and HD-compatible content became available, matrix coefficients defined by ITU-R BT.709 came to be used. On the other hand, general personal computers originally dealt mostly with still images and graphics, and as specified in Exif JPEG recording, matrix coefficients defined by ITU-R BT.601 and range settings conventionally used as full range were often employed. Subsequently, images with HD pixel counts or higher began to be frequently used on personal computers and other devices, and a diverse range of operating methods coexisted faster than industry standards could be established.

[0005] Ideally, any system inconsistencies should be treated as bugs and addressed. However, in recent development of image equipment and software, the reuse of existing libraries and processors has made them easier to use even for non-experts. Furthermore, errors in matrix coefficients and range settings, even if incorrect, are still visible in the image, making them difficult to perceive as bugs at first glance, and thus hindering countermeasures. The expansion of network usage, along with the involvement of relay processing and reuse, has also made it difficult to identify and address areas where standard inconsistencies occur. Thus, the market is a mix of diverse standards and operations, and the correct transmission of standards between devices cannot be relied upon. As a result, the color reproduction of image signals can be impaired, and addressing this issue is a challenge.

[0006] This invention has been made in view of the above circumstances, and one of its objectives is to provide an information processing system, an information processing device, and a control method that can suppress the deterioration of color reproduction of image signals caused by the coexistence of various standards in a simple manner. [Means for solving the problem]

[0007] The present invention has been made to solve the above problems, and an information processing system according to a first aspect of the present invention comprises an imaging device and an information processing device that acquires an image signal output from the imaging device and performs processing to display an image based on the acquired image signal on a display unit, wherein the imaging device comprises an imaging unit that outputs an imaging signal of an image of a subject, a test signal output unit that outputs a preset test image signal, and an imaging processing unit that encodes the imaging signal output from the imaging unit or the test image signal output from the test signal output unit according to a first standard set in the imaging device and outputs an image signal after encoding, and the information processing device encodes the image signal output from the imaging device according to a second standard The system comprises: an image processing unit that decodes the image signal and outputs the decoded image signal; a detection unit that acquires the image signal obtained from the imaging device after the image processing unit has decoded the encoded image signal, and detects an inconsistency between the first standard and the second standard based on the signal level of the decoded image signal obtained; and a correction unit that, if the detection unit has detected an inconsistency between the first standard and the second standard, determines a correction content to correct the inconsistency between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit, wherein the correction unit performs a correction process by applying the correction content to the encoded image signal obtained from the imaging device.

[0008] In the above-described information processing system, each level of the image signal obtained by encoding the test image signal with the first standard and then decoding it with the second standard using a combination of multiple types of the first standard and multiple types of the second standard is pre-set as one of several detection patterns used by the detection unit to detect inconsistencies between the first standard and the second standard, and the detection unit may detect inconsistencies between the first standard and the second standard by comparing the image signal obtained by decoding the encoded image signal of the test image signal acquired from the imaging device with one of the multiple detection patterns.

[0009] In the above information processing system, for each of the detection patterns in a combination where the first standard and the second standard are inconsistent among the plurality of detection patterns, respective correction details for correcting the inconsistency between the first standard and the second standard are associated and set. When the correction unit detects an inconsistency between the first standard and the second standard by the detection unit, the correction unit may perform the correction process by applying the correction details associated with the corresponding detection pattern among the plurality of detection patterns.

[0010] In the above information processing system, the first standard and the second standard may include standards for color spaces, and the correction details may include a conversion formula when converting the color space.

[0011] In the above information processing system, the first standard and the second standard may include standards for the number of gray levels, and the correction details may include a conversion formula when converting the number of gray levels.

[0012] In the above information processing system, when detecting an inconsistency between the first standard and the second standard, the detection unit transmits an output instruction for the test image signal to the imaging device, and when receiving the output instruction for the test image signal, the imaging processing unit encodes the test image signal output from the test signal output unit according to the first standard and outputs the encoded image signal.

[0013] Furthermore, an information processing device according to a second aspect of the present invention, which selects either an imaging signal capturing an image of a subject or a preset test image signal, encodes it according to a first standard set in the imaging device, and outputs an encoded image signal, and which acquires an image signal output from the imaging device and performs a process of displaying a display image based on the acquired image signal on a display unit, comprises: an image processing unit that decodes the image signal output from the imaging device according to a second standard and outputs a decoded image signal; a detection unit that acquires an image signal after the image processing unit decodes the encoded image signal of the test image signal acquired from the imaging device by instructing the imaging device to output the test image signal, and detects an inconsistency between the first standard and the second standard based on the signal level of the acquired decoded image signal; and a correction unit that, when the detection unit detects an inconsistency between the first standard and the second standard, determines a correction content to correct the inconsistency between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit, wherein the correction unit performs a correction process by applying the correction content to the encoded image signal of the imaging signal acquired from the imaging device.

[0014] Furthermore, a control method for an information processing device according to a third aspect of the present invention, which selects either an imaging signal capturing an image of a subject or a preset test image signal, encodes it according to a first standard set in the imaging device, and outputs an image signal after encoding, and which acquires an image signal output from the imaging device and performs a process of displaying a display image based on the acquired image signal on a display unit, includes the steps of: an image processing unit decodes the image signal output from the imaging device according to a second standard and outputs an image signal after decoding; a detection unit acquires an image signal after the image processing unit decodes the image signal after encoding the test image signal acquired from the imaging device by instructing the imaging device to output the test image signal, and detects an inconsistency between the first standard and the second standard based on the signal level of the acquired decoded image signal; a correction unit, when an inconsistency between the first standard and the second standard is detected by the detection unit, determines a correction content to correct the inconsistency between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit; and a correction unit performs a correction process by applying the correction content to the image signal after encoding the imaging signal acquired from the imaging device. [Effects of the Invention]

[0015] According to the above-described aspect of the present invention, it is possible to suppress, in a simple manner, the degradation of color reproduction of image signals caused by the coexistence of various standards. [Brief explanation of the drawing]

[0016] [Figure 1] A diagram showing an example of an information processing system according to the first embodiment. [Figure 2] A block diagram showing an example of the hardware configuration of an information processing system according to the first embodiment. [Figure 3] A block diagram showing an example of the functional configuration of an information processing system according to the first embodiment. [Figure 4] A figure showing an example of a test image signal used to detect a standard mismatch according to the first embodiment. [Figure 5] A figure showing an example of a detection pattern for mismatch in the matrix coefficients of a color space according to the first embodiment. [Figure 6] A figure showing another example of a detection pattern for mismatches in the matrix coefficients of a color space according to the first embodiment. [Figure 7] A figure showing an example of a detection pattern for range mismatch according to the first embodiment. [Figure 8] A figure showing an example of the determination and correction content for matrix coefficient mismatch for each detection pattern according to the first embodiment. [Figure 9] A diagram showing an example of the determination and correction content for range mismatch for each detection pattern according to the first embodiment. [Figure 10] A flowchart showing an example of the color reproduction compensation process on the camera side according to the first embodiment. [Figure 11] A flowchart showing an example of color reproduction compensation processing on the information processing device side according to the first embodiment. [Figure 12] A configuration diagram showing an example of an information processing system 1A according to the second embodiment. [Modes for carrying out the invention]

[0017] Embodiments of the present invention will now be described with reference to the drawings. An information processing system according to an embodiment of the present invention comprises a camera and an information processing device that acquires an image signal output from the camera and performs processing to display an image based on the acquired image signal. The camera may be built into the information processing device or it may be an external device (peripheral device) connected to the information processing device.

[0018] <First Embodiment> First, a first embodiment of the present invention will be described. Figure 1 is a configuration diagram showing an example of the information processing system 1 according to this embodiment. As shown in Figure 1, the information processing system 1 according to this embodiment is configured as an information processing device 10 with a built-in camera 50. The information processing device 10 is an example of a computer device, for example, a clamshell-type (notebook-type) PC (Personal Computer). The camera 50 is provided in the information processing device 10 as an example of an imaging device for capturing images of a subject. The information processing device 10 displays the image captured by the camera 50 on the display unit 15.

[0019] (Hardware configuration of the information processing device 10) Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 10 according to this embodiment. The illustrated information processing device 10 includes a display unit 15, a USB connector 16, a communication unit 17, a storage unit 18, an input unit 19, an EC (Embedded Controller) 20, a power supply unit 21, a battery 22, and a system processing unit 100.

[0020] The display unit 15 includes a liquid crystal display (LCD) or an organic EL (electroluminescence) display. The display unit 15 displays an image based on display data based on the control of the system processing unit 100. The display data includes still images, moving images, and text data acquired or generated by the OS processing or the processing of an application running on the OS. For example, as shown in Figure 1, when the camera 50 and the information processing device 10 are connected, the display unit 15 displays an image based on the image signal acquired from the camera 50 based on the processing of an application or the like.

[0021] USB connector 16 is a connection terminal for connecting peripheral devices that utilize USB (Universal Serial Bus). For example, USB connector 16 is a receptacle-side terminal compliant with the USB Type-C standard.

[0022] The communication unit 17 connects to other devices via a wireless or wired communication network and transmits and receives various types of data. For example, the communication unit 17 is configured to include a wired LAN interface such as Ethernet® and a wireless LAN interface such as Wi-Fi®.

[0023] The storage unit 18 is comprised of storage media such as an HDD (Hard Disk Drive), SSD (Solid State Drive), RAM (Random Access Memory), and ROM (Read Only Memory). For example, the storage unit 18 stores programs such as the OS, various drivers, various services / utilities, and applications, as well as various data.

[0024] The input unit 19 is an input unit that receives user input and is configured to include a keyboard, for example, as shown in the information processing device 10 in Figure 1. The input unit 19 receives user operations on the keyboard and outputs an operation signal corresponding to the user's operation to the EC20. The input unit 19 may also be configured to include a touch panel, touchpad, etc., in addition to or instead of a keyboard. Furthermore, the input unit 19 may be connected by wired or wireless connection to an external operating device such as a mouse or an external keyboard, and may receive user operations on the connected external operating device.

[0025] The EC20 is a one-chip microcomputer that monitors and controls various devices (peripherals, sensors, etc.) regardless of the OS system state. The EC20 is composed of a CPU (Central Processing Unit), RAM, ROM, etc. (not shown), and is equipped with multiple A / D input terminals, D / A output terminals, a timer, and digital input / output terminals. The input unit 19, power supply unit 21, and system processing unit 100 are connected to the EC20 via these input / output terminals. The EC20 receives or transmits various signals to each of the connected units.

[0026] For example, EC20 acquires operation signals output from the input unit 19 and executes processing based on the acquired operation signals. EC20 outputs operation signals related to the processing of the system processing unit 100 to the system processing unit 100. EC20 also controls the power supply unit 21 according to the system state of the OS, etc. For example, EC20 outputs control signals to the power supply unit 21 to control the power supply according to the system state, etc. Furthermore, EC20 acquires information about the state of the battery 22 (such as remaining capacity) from the power supply unit 21 by communicating with it.

[0027] The power supply unit 21 includes, for example, a DC / DC converter and a charge / discharge circuit for controlling the charging or discharging of the battery 22. The power supply unit 21 converts the DC power supplied from the battery 22 or from an external power supply (not shown, such as an AC adapter) into multiple voltages necessary to operate each part of the information processing device 10. The power supply unit 21 supplies power to each part of the information processing device 10 based on the control of the EC20.

[0028] Battery 22 is a secondary battery that supplies power to each part of the information processing device 10 when power is not supplied from an external power source (such as an AC adapter). When power is supplied from an external power source (such as an AC adapter), battery 22 is charged by that power via the power supply unit 21 until it is fully charged. When power is not supplied from an external power source (such as an AC adapter), the power stored in battery 22 is discharged and supplied to each part of the information processing device 10 via the power supply unit 21.

[0029] The system processing unit 100 consists of a CPU 101, a GPU (Graphics Processing Unit) 102, a memory controller 103, an I / O (Input-Output) controller 104, and system memory 105. The CPU 101 and GPU 102 are sometimes collectively referred to as the processor.

[0030] The CPU 101 executes processing by programs such as the OS, various drivers, various services / utilities, and applications. As an example of the OS, Windows (registered trademark) is used. The GPU 102 is connected to the display unit 15. The GPU 102 performs image processing based on the control of the CPU 101 and generates display data. The GPU 102 outputs the generated display data to the display unit 15. The CPU 101 and GPU 102 may be integrated into a single core, or the load may be shared between the CPU 101 and GPU 102, which are formed as individual cores. The number of processors is not limited to one; there may be multiple processors.

[0031] The memory controller 103 controls the reading and writing of data from the system memory 105 and storage unit 18, etc., by the CPU 101 and GPU 102. The I / O controller 104 controls the input and output of data to and from the display unit 15, USB connector 16, communication unit 17, EC20, camera 50, etc.

[0032] System memory 105 is writable memory used as a reading area for programs executed by processors such as the CPU 101 and GPU 102, or as a work area for writing processing data for such programs. For example, system memory 105 is composed of multiple DRAM (Dynamic Random Access Memory) chips. Programs include the OS, various drivers for controlling peripheral devices, various service / utility programs, and application programs.

[0033] Camera 50 outputs an image signal based on the imaging signal of the subject. For example, camera 50 is connected to the I / O controller 104 via USB. Note that camera 50 is not limited to a built-in camera with a USB connection, but may also be a built-in camera with a MIPI (Mobile Industry Processor Interface) connection, etc.

[0034] (Image signal correction processing) Next, we will explain the image signal correction process performed when displaying the image captured by the camera 50 on the display unit 15 of the information processing device 10. Figure 3 is a block diagram showing an example of the functional configuration of the information processing system 1 according to this embodiment. This figure shows an example of the functional configuration of the information processing system 1 that performs image signal correction processing. In Figure 3, the same reference numerals are used for components corresponding to Figures 1 and 2.

[0035] The camera 50 comprises an imaging unit 51, an image processing unit 52, and a test signal generator 53. The imaging unit 51 includes a lens and an image sensor (not shown) and outputs an image signal of a subject to the image processing unit 52. The image processing unit 52 encodes the image signal output from the imaging unit 51 according to a standard set in the camera 50 and outputs the encoded image signal.

[0036] The information processing device 10 includes a camera driver 110, an application processing unit 111, a detection unit 112, and a correction unit 113, as a functional configuration realized by the system processing unit 100 executing programs such as applications and drivers. The system processing unit 100 acquires the encoded image signal output from the camera 50 via the I / O controller 104 (see Figure 2), decodes the acquired image signal, and displays it on the display unit 15.

[0037] The camera driver 110 is a functional configuration realized by the execution of a program that enables the camera 50 to be used. For example, the camera driver 110 decodes the image signal acquired from the camera 50 and makes it available for use by the application.

[0038] The application processing unit 111 is a functional configuration realized by the execution of various application programs. For example, in the processing of an application that processes image signals acquired from the camera 50, the application processing unit 111 performs image processing on the decoded image signal output from the camera driver 110. At this time, the application processing unit 111 outputs the decoded image signal and the image signal after image processing output from the camera driver 110 to the display unit 15. In other words, the image signal acquired from the camera 50 by the system processing unit 100 is sent to the display unit 15 via the camera driver 110 and the application processing unit 111, and the image based on the decoded image signal is displayed on the display unit 15.

[0039] The standard used for decoding by the information processing device 10 (system processing unit 100) depends on the hardware or software that performs the decoding within the information processing device 10. Here, a configuration in which the camera driver 110 performs the decoding is explained as an example, but the application (application processing unit 111) may also perform the decoding. Alternatively, the display driver that controls the display unit 15 may perform the decoding. Furthermore, the functions of the camera driver 110 may be implemented in hardware, and the decoding may be performed in hardware. Note that the encoding or decoding of the image signal may be performed multiple times before it is displayed on the display unit 15. For example, the image signal may be decoded by both the camera driver 110 and the application processing unit 111.

[0040] This section explains the standards for encoding and decoding image signals. The still or moving image signals handled by the information processing device 10 and the camera 50 are converted from the YUV color space to the RGB color space or from the RGB color space to the YUV color space according to a predetermined standard (format). There are several types of standards, such as "ITU-R BT.601," which is the standard for older television broadcasting (SDTV: Standard Definition Television), and "ITU-R BT.709," which is the standard for HDTV (High-Definition Television). These various types of standards differ depending on the specifications of the camera 50 and the hardware or software specifications of the information processing device 10, resulting in a situation where multiple types of standards coexist in the market.

[0041] If the encoding standard used by camera 50 and the decoding standard used by information processing device 10 are different, the matrix coefficients for color space conversion will differ, which may result in loss of color reproduction when the information processing device 10 displays the image captured by camera 50. For example, if an image signal encoded with the color space conversion matrix coefficients defined in "ITU-R BT.601" is decoded using the same standard, the color information will generally be converted correctly.

[0042] On the other hand, when an image signal encoded with the color space conversion matrix coefficients defined in "ITU-R BT.601" is decoded with the color space conversion matrix coefficients defined in a different standard, "ITU-R BT.709," the color information is not converted correctly. Similarly, when an image signal encoded with the color space conversion matrix coefficients defined in "ITU-R BT.709" is decoded with the color space conversion matrix coefficients defined in a different standard, "ITU-R BT.601," the color information is not converted correctly.

[0043] Furthermore, when displaying 8-bit image signals, there are two ranges (number of gradations): full range, which displays with 256 gradations from "0 to 255," and limited range, which displays with 220 gradations limited to "16 to 235." In full range, the range from "0 to 255" corresponds to the color space, while in limited range, the range from "16 to 235" corresponds to the color space. Therefore, it is desirable to decode image signals encoded in full range in full range, and similarly, it is desirable to decode image signals encoded in limited range in limited range.

[0044] When a range mismatch occurs, such as decoding a full-range encoded image signal with a limited range, or decoding a limited-range encoded image signal with a full range, the RGB gradation may not be converted correctly, and as a result, the color information may not be converted correctly.

[0045] For example, if the image signal output from camera 50 has the correct standard information attached, it is possible to know which standard it was encoded in. However, in reality, incorrect standard information may be attached, or standard information may not be attached at all. While the ideal solution would be to promote the correct application of standards, this is unlikely to be feasible or even implemented quickly in environments such as PCs that rely on many existing devices.

[0046] Therefore, the information processing system 1 according to this embodiment does not rely on standard information, but instead detects and corrects errors in the conversion of image signals caused by standard inconsistencies. As mentioned above, multiple types of standards are mixed, but the error patterns after conversion of image signals when standard inconsistencies occur can be estimated. For this reason, the information processing system 1 can use the image signal of a specific test image (test image) (referred to as the "test image signal") to compare the image signal after encoding and decoding with the error pattern and determine the detection and correction content of standard inconsistencies. For example, in order to detect standard inconsistencies (mismatches) between encoding and decoding, the information processing system 1 prepares detection patterns that include error patterns in advance and detects standard inconsistencies.

[0047] Refer to Figures 4-6 to explain the detection patterns for detecting inconsistencies in the matrix coefficients of the color space. Figure 4 shows an example of a test image signal used to detect standard inconsistencies according to this embodiment. Figure 4(A) is an example of a test image signal used to detect standard inconsistencies. Here, a 75% color bar is used as the test image. That is, a 75% color bar image signal is used as the test image signal.

[0048] Figure 4(B) is a graph showing the RGB levels (8-bit data values) of the test image signal. If the RGB levels of the image signal after encoding and decoding the test image signal are in the state shown in Figure 4(B), it can be determined that the encoding matrix coefficients and the decoding matrix coefficients match, that is, that the encoding matrix coefficients and the decoding matrix coefficients are consistent. In other words, the RGB levels shown in Figure 4(B) can be used as a detection pattern to detect that the encoding and decoding matrix coefficients are consistent. The ID indicating this process in which the encoding and decoding matrix coefficients are consistent is denoted as "M0".

[0049] Figure 5 shows an example of a detection pattern for color space matrix coefficient mismatch according to this embodiment. Figure 5(A) is a graph showing the RGB levels (8-bit data values) when a test image signal (75% color bar) is encoded with "ITU-R BT.709" and decoded with "ITU-R BT.601". If the RGB levels of the image signal after encoding and decoding the test image signal are in the state shown in Figure 5(A), it can be determined that the standards are mismatched. In other words, the RGB levels shown in Figure 5(A) are a detection pattern for detecting matrix coefficient mismatch when encoding is "ITU-R BT.709" and decoding is "ITU-R BT.601", and the ID indicating this encoding and decoding process is "Ma1".

[0050] Figure 5(B) is a graph showing the RGB levels (8-bit data values) when the "Ma1" process in Figure 5(A) is performed twice. In other words, the RGB levels shown in Figure 5(B) are a detection pattern that detects inconsistencies in matrix coefficients when encoding by "ITU-R BT.709" and decoding by "ITU-R BT.601" are performed twice, and the ID indicating the process of performing this "Ma1" process twice is denoted as "Ma2".

[0051] Figure 6 shows another example of a detection pattern for color space matrix coefficient mismatch according to this embodiment. Figure 6(A) is a graph showing the RGB levels (8-bit data values) when a test image signal (75% color bar) is encoded with "ITU-R BT.601" and decoded with "ITU-R BT.709". If the RGB levels of the image signal after encoding and decoding the test image signal are in the state shown in Figure 6(A), it can be determined that the standards are mismatched. In other words, the RGB levels shown in Figure 6(A) are a detection pattern for detecting matrix coefficient mismatch when encoding is "ITU-R BT.601" and decoding is "ITU-R BT.709", and the ID indicating this encoding and decoding process is "Mb1".

[0052] Figure 6(B) is a graph showing the RGB levels (8-bit data values) when the "Mb1" process in Figure 6(A) is performed twice. In other words, the RGB levels shown in Figure 6(B) are a detection pattern that detects inconsistencies in matrix coefficients when encoding by "ITU-R BT.601" and decoding by "ITU-R BT.709" are performed twice, and the ID indicating the process of performing this "Mb1" process twice is denoted as "Mb2".

[0053] Next, with reference to Figure 7, a detection pattern for detecting range mismatch will be described. Figure 7 shows an example of a range mismatch detection pattern according to this embodiment. When detecting range mismatch, for example, the RGB level (8-bit data value) of white in the test image signal (75% color bar) shown in Figure 4(A) is used.

[0054] The leftmost graph in Figure 7 shows the RGB level (8-bit data value) of white in the test image signal (75% color bar). When the test image signal is encoded and decoded, if the RGB level of white in the decoded image signal is at this state (the same level as the test image signal), it can be determined that the encoding range and the decoding range match, that is, the encoding range and the decoding range are consistent. The ID indicating this consistency between encoding and decoding ranges is denoted as "R0". In other words, the RGB level shown as "R0" can be used as a detection pattern to detect that the encoding and decoding ranges are consistent.

[0055] Furthermore, the ID for the process of encoding in full range and decoding in limited range is denoted as "Ra". Here, the ID for the process of performing the full-range encoding and limited-range decoding process once is denoted as "Ra1", and the ID for the process of performing the "Ra1" process twice is denoted as "Ra2". In other words, the RGB level of white shown as "Ra1" is a detection pattern for range inconsistency when full-range encoding and limited-range decoding are performed once. Similarly, the RGB level of white shown as "Ra2" is a detection pattern for range inconsistency when full-range encoding and limited-range decoding are performed twice.

[0056] Furthermore, the ID for the process of encoding in limited range and decoding in full range is denoted as "Rb". Here, the ID for the process of encoding in full range and decoding in limited range once is denoted as "Rb1", and the ID for the process of performing the "Rb1" process twice is denoted as "Rb2". In other words, the RGB level of white shown in "Rb1" is a detection pattern for range inconsistency when encoding in limited range and decoding in full range once is performed. Similarly, the RGB level of white shown in "Rb2" is a detection pattern for range inconsistency when encoding in limited range and decoding in full range twice is performed.

[0057] Next, we will explain the criteria for detecting matrix coefficient mismatches and range mismatches, as well as the correction methods used when mismatches are detected.

[0058] First, the determination and correction of matrix coefficient mismatches will be explained with reference to Figure 8. Figure 8 is a diagram showing an example of the determination and correction of matrix coefficient mismatches for each detection pattern according to this embodiment. In Figure 8, the IDs of each process ("M0", "Ma1", "Ma2", "Mb1", "Mb2") explained with reference to Figures 4 to 6, each detection pattern (the content of each process), the estimated level of the image signal output by each process, and the correction content are shown in relation to each other.

[0059] "M0" is a detection pattern for a process where the matrix coefficients of encoding and decoding match (are identical). At this time, the estimated levels of RGB of the output image signal (i.e., the RGB levels of the test image signal) "v , , -1 , , Ma1 , , ,

[0062] , , , , , 709 , , 601 , , Ma1 , , ,

[0065] , ,

[0064] , , <00​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

[0066] In other words, in the case of a mismatch in matrix coefficients due to the "Ma1" processing, the RGB levels of the image signal to be corrected are set to "f Ma1 This can be corrected using the inverse function of ( ).

[0067] Next, "Ma2" is a detection pattern that performs the "Ma1" process twice, and is the estimated level of the RGB of the output image signal "v Ma2i This can be expressed as equation 4 below.

[0068]

number

[0069] Note that "f Ma2 The function "()" calculates the output RGB levels when the "Ma1" process is applied twice to the RGB levels of the test image signal. Here, "f Ma2 ()=((M 601 ) -1 M 709 ) 2 It is defined as ".

[0070] In the case of matrix coefficient mismatch due to this "Ma2" processing, the correction can be expressed by the following equation 5, where "v0" is the RGB level of the image signal to be corrected.

[0071]

number

[0072] In other words, in the case of a mismatch in matrix coefficients due to the "Ma2" processing, the RGB levels of the image signal to be corrected are set to "f Ma2 This can be corrected using the inverse function of ( ).

[0073] Next, "Mb1" is a detection pattern for a process where encoding is "ITU-R BT.601" and decoding is "ITU-R BT.709", and the estimated RGB levels of the output image signal are "v Mb1i This can be expressed as equation 6 below.

[0074]

number

[0075] Note that "f Mb1 The function "()" calculates the output RGB levels when the "Mb1" process is applied to the RGB levels of the test image signal. Here, "f Mb1 ()=(M 709 ) -1 M 601 It is defined as ".

[0076] In the case of a mismatch in matrix coefficients due to this "Mb1" processing, the correction can be expressed by the following equation 7, where "v0" is the RGB level of the image signal to be corrected.

[0077]

number

[0078] In other words, in the case of a mismatch in matrix coefficients due to the processing of "Mb1", the RGB levels of the image signal to be corrected are set to "f Mb1 This can be corrected using the inverse function of ( ).

[0079] Next, "Mb2" is a detection pattern of processing that performs the same process as "Mb1" twice, and is the estimated level of the RGB of the output image signal "v Mb2i This can be expressed as equation 8 below.

[0080]

number

[0081] Note that "fMb2 The function "()" calculates the output RGB levels when the "Mb1" process is applied twice to the RGB levels of the test image signal. Here, "f Mb2 ()=((M 709 ) -1 M 601 ) 2 It is defined as ".

[0082] In the case of matrix coefficient mismatch due to this "Mb2" processing, the correction can be expressed by the following equation 9, where "v0" is the RGB level of the image signal to be corrected.

[0083]

number

[0084] In other words, in the case of a mismatch in matrix coefficients due to the processing of "Mb2", the RGB levels of the image signal to be corrected are set to "f Mb2 This can be corrected using the inverse function of ( ).

[0085] Next, the criteria for determining and correcting range mismatches will be explained with reference to Figure 9. Figure 9 is a diagram showing an example of the criteria for determining and correcting range mismatches for each detection pattern according to this embodiment. In Figure 9, the IDs of each process ("R0", "Ra1", "Ra2", "Rb1", "Rb2") explained with reference to Figure 7, each detection pattern (the content of each process), the estimated level of the image signal output by each process, and the correction content are shown in relation to each other.

[0086] "R0" is a detection pattern of a process where the encoding and decoding ranges are matched (coordinated), and it represents the estimated level of the white RGB of the output image signal at this time (i.e., the white RGB level of the test image signal) "N Iw This can be expressed by the following equation 10.

[0087]

number

[0088] "Ra1" is a detection pattern for a process where encoding is full-range and decoding is limited-range. Since the full-range gradation is "0-255" and the limited-range gradation is "16-235", the estimated RGB level of the output image signal is "N Ra1 This can be expressed in the following equation 11.

[0089]

number

[0090] Note that "f Ra1 The function "()" calculates the output RGB levels when the "Ra1" process is applied to the RGB levels of the test image signal, and is defined as shown in equation 11 above.

[0091] In the case of range mismatch due to this "Ra1" processing, the correction can be expressed by the following equation 12, where "N0" is the RGB level of the image signal to be corrected.

[0092]

number

[0093] In other words, in the case of range mismatch due to "Ra1" processing, the RGB levels of the image signal to be corrected are set to "f Ra1 This can be corrected using the inverse function of ( ).

[0094] Next, "Ra2" is a detection pattern that performs the "Ra1" process twice, and is the estimated level "N" of the RGB of the output image signal. Ra2 This can be expressed in the following equation 13.

[0095]

number

[0096] Note that "fRa2 The function "()" calculates the output RGB levels when the "Ra1" process is applied twice to the RGB levels of the test image signal, and is defined as shown in equation 13 above.

[0097] In the case of range mismatch due to this "Ra2" processing, the correction can be expressed by the following equation 14, where "N0" is the RGB level of the image signal to be corrected.

[0098]

number

[0099] In other words, in the case of range mismatch due to "Ra2" processing, the RGB levels of the image signal to be corrected are set to "f Ra2 This can be corrected using the inverse function of ( ).

[0100] Next, "Rb1" is the detection pattern for a process where encoding is limited range and decoding is full range, and the estimated level of RGB in the output image signal is "N Rb1 This can be expressed in the following equation 15.

[0101]

number

[0102] Note that "f Rb1 The function "()" calculates the output RGB levels when the "Rb1" process is applied to the RGB levels of the test image signal, and is defined as shown in equation 15 above.

[0103] In the case of range mismatch due to this "Rb1" processing, the correction can be expressed by the following equation 16, where "N0" is the RGB level of the image signal to be corrected.

[0104]

number

[0105] In other words, in the case of range mismatch due to processing of "Rb1", the RGB levels of the image signal to be corrected are set to "f Rb1 This can be corrected using the inverse function of ( ).

[0106] Next, "Rb2" is a detection pattern of processing that performs the "Rb1" process twice, and is the estimated level "N" of the RGB of the output image signal. Rb2 This can be expressed in the following equation 17.

[0107]

number

[0108] Note that "f Rb2 The function "()" calculates the output RGB levels when the "Rb1" process is applied twice to the RGB levels of the test image signal, and is defined as shown in equation 17 above.

[0109] In the case of range mismatch due to this "Rb2" processing, the correction can be expressed by the following equation 18, where "N0" is the RGB level of the image signal to be corrected.

[0110]

number

[0111] In other words, in the case of range mismatch due to "Rb2" processing, the RGB levels of the image signal to be corrected are set to "f Rb2 This can be corrected using the inverse function of ( ).

[0112] Next, returning to Figure 3, we will describe a configuration that detects and corrects inconsistencies in the standards (color space matrix coefficients and ranges) explained with reference to Figures 4-9. Since this process aims to suppress the loss of color reproduction in the image signal, it will be referred to as "color reproduction compensation processing" in the following explanation.

[0113] The test signal generator 53 in camera 50 is configured to output a test image signal used for color reproduction compensation processing. For example, the test signal generator 53 outputs a 75% color bar image signal as shown in Figure 4 as a test image signal.

[0114] The imaging processing unit 52 can encode not only the imaging signal output from the imaging unit 51, but also the test image signal output from the test signal generator 53, according to the standard set in the camera 50, and output the encoded image signal. In normal processing, the imaging processing unit 52 encodes and outputs the imaging signal, but when it receives an instruction from the information processing device 10 to output a test image signal, it switches to the process of encoding and outputting the test image signal.

[0115] For example, the test signal generator 53 outputs a test image signal in response to instructions from the imaging processing unit 52, only when the imaging processing unit 52 is encoding and outputting a test image signal.

[0116] Furthermore, the detection unit 112 and correction unit 113 of the information processing device 10 are the main components that perform color reproduction compensation processing on the information processing device 10 side. The detection unit 112 acquires the decoded image signal from the encoded image signal of the test image signal acquired from the camera 50, and detects inconsistencies between the encoding standard and the decoding standard based on the signal level of the acquired decoded image signal. For example, if the detection unit 112 detects an inconsistency in the standards, it instructs the correction unit 113 to perform a correction.

[0117] If the detection unit 112 detects an inconsistency between the encoding standard and the decoding standard, the correction unit 113 determines a correction to correct the inconsistency between the encoding standard and the decoding standard based on the signal level of the decoded image signal acquired by the detection unit 112, and performs a correction process by applying the said correction to the image signal acquired from the camera 50. The processes performed by the detection unit 112 and the correction unit 113 will be described in detail below.

[0118] First, when performing color reproduction compensation processing, the detection unit 112 sends an instruction to the camera 50 to output a test image signal. For example, the detection unit 112 sends an instruction to the camera 50 to output a test image signal when a user launches a specific application, when a user gives an instruction via a specific UI, when the application environment is reset or updated, or when the application is launched for the first time after that. The detection unit 112 then performs a detection process for inconsistencies in the encoding and decoding standards of the image signal.

[0119] Specifically, the detection unit 112 acquires the decoded image signal from the encoded image signal of the test image signal acquired from the camera 50, and detects an inconsistency between the encoding standard and the decoding standard based on the signal level of the acquired decoded image signal.

[0120] For example, when the detection unit 112 acquires the decoded image signal, it captures the display screen of the display unit 15, which displays the image based on the decoded image signal, and then detects and extracts the portion of the test image from the captured image.

[0121] Furthermore, the detection unit 112 detects inconsistencies between the encoding standard and the decoding standard by comparing the acquired decoded image signal with multiple detection patterns. The multiple detection patterns are those used to detect inconsistencies in the matrix coefficients and ranges of the color space, as explained with reference to Figures 4-9.

[0122] For example, regarding the detection of matrix coefficient mismatches, as shown in Figure 8, the levels of the image signals (estimated levels of the output image signals) obtained when encoding and then decoding a test image signal using combinations of encoding matrix coefficients and decoding matrix coefficients (ID: "M0", "Ma1", "Ma2", "Mb1", "Mb2") are each pre-set as multiple detection patterns used to detect matrix coefficient mismatches.

[0123] The detection unit 112 then compares the RGB levels of the decoded image signal with multiple detection patterns (estimated levels of the output image signal), determines that the encoding matrix coefficients and decoding matrix coefficients of the closest (smallest difference) detection pattern are in line with the current standard, and determines whether the two matrix coefficients are compatible or incompatible.

[0124] For example, the detection unit 112 determines that the encoding matrix coefficients and the decoding matrix coefficients are in agreement if the detection pattern for "M0" is the closest (smallest difference). Conversely, the detection unit 112 determines that the encoding matrix coefficients and the decoding matrix coefficients are in agreement if the detection patterns for "Ma1", "Ma2", "Mb1", or "Mb2" are the closest (smallest difference).

[0125] Furthermore, as shown in Figure 8, for each of the detection patterns among the multiple detection patterns in which the encoding matrix coefficients and the decoding matrix coefficients are inconsistent (ID: "Ma1", "Ma2", "Mb1", "Mb2"), a corresponding correction content for correcting the inconsistency is associated and set. When the detection unit 112 detects an inconsistency between the encoding matrix coefficients and the decoding matrix coefficients, the correction unit 113 performs a correction process by applying the correction content associated with the corresponding detection pattern among the multiple detection patterns.

[0126] On the other hand, regarding the detection of range mismatch (full range and limited range), as shown in Figure 9, the levels of the image signals (estimated levels of the output image signals) obtained when encoding and then decoding a test image signal using combinations of encoding range and decoding range (ID: R0, Ra1, Ra2, Rb1, Rb2) are each pre-set as multiple detection patterns used to detect range mismatch.

[0127] The detection unit 112 then compares the RGB levels of the decoded image signal with multiple detection patterns (estimated levels of the output image signal), determines that the encoding range and decoding range of the closest (smallest difference) detection pattern are in line with the current standard, and determines whether the two ranges are compatible or incompatible.

[0128] For example, the detection unit 112 determines that the encoding range and the decoding range are compatible if the detection pattern for "R0" is the closest (smallest difference). Conversely, the detection unit 112 determines that the encoding range and the decoding range are incompatible if the detection patterns for "Ra1", "Ra2", "Rb1", or "Rb2" are the closest (smallest difference).

[0129] Furthermore, as shown in Figure 9, for each of the detection patterns among the multiple detection patterns in which the encoding range and the decoding range are inconsistent (ID: "Ra1", "Ra2", "Rb1", "Rb2"), a corresponding correction content for correcting the inconsistency is associated and set. When the detection unit 112 detects an inconsistency between the encoding range and the decoding range, the correction unit 113 performs a correction process by applying the correction content associated with the corresponding detection pattern from among the multiple detection patterns.

[0130] (Operation of color reproduction compensation processing) Next, we will explain the operation of the color reproduction compensation process performed by the information processing system 1. First, the operation of the color reproduction compensation process on the camera 50 side will be explained with reference to Figure 10. Figure 10 is a flowchart showing an example of the color reproduction compensation process on the camera side according to this embodiment.

[0131] (Step S101) In camera 50, the imaging processing unit 52 determines whether or not it has received an instruction from the information processing unit 10 to output a test image signal. If the imaging processing unit 52 determines that it has received an instruction from the information processing unit 10 to output a test image signal (YES), it proceeds to step S103. On the other hand, if the imaging processing unit 52 determines that it has not received an instruction from the information processing unit 10 to output a test image signal (NO), it proceeds to step S105.

[0132] (Step S103) The imaging processing unit 52 outputs an image signal encoded from the test image signal (75% color bar) output from the test signal generator 53.

[0133] (Step S105) The imaging processing unit 52 outputs an image signal encoded from the imaging signal captured by the imaging unit 51.

[0134] Next, the operation of the color reproduction compensation process on the information processing device 10 will be explained with reference to Figure 11. Figure 11 is a flowchart showing an example of the color reproduction compensation process on the information processing device side according to this embodiment.

[0135] (Step S201) In the information processing device 10, the detection unit 112 sends an instruction to the camera 50 to output a test image signal when performing color reproduction compensation processing. Then, the process proceeds to step S203.

[0136] (Step S203) The camera driver 110 acquires an image signal encoded with the test image signal from the camera 50, decodes the acquired image signal, and makes it usable by the application. The application processing unit 111 outputs the decoded image signal output from the camera driver 110 to the display unit 15 and displays it on the display unit 15. Then, the process proceeds to step S205.

[0137] (Step S205) The detection unit 112 acquires the decoded image signal. For example, the detection unit 112 captures the display screen of the display unit 15.

[0138] (Step S207) The detection unit 112 determines whether or not it detected the test image (75% color bar) from the screen captured image in step S205. If the detection unit 112 determines that it did not detect the test image from the screen captured image (step S207: NO), it returns to the process in step S205. On the other hand, if the detection unit 112 determines that it did detect the test image from the screen captured image (step S207: YES), it proceeds to the process in step S209.

[0139] (Step S209) The detection unit 112 determines whether the encoding and decoding standards are consistent or inconsistent. For example, if the detection unit 112 detects an inconsistency in matrix coefficients, it compares the RGB levels of the decoded image signal with the multiple detection patterns shown in Figure 8 (estimated levels of the output image signal), determines that the encoding matrix coefficients and decoding matrix coefficients of the closest (smallest difference) detection pattern are in line with the current standard, and determines whether the two matrix coefficients are consistent or inconsistent. Also, if the detection unit 112 detects a range inconsistency, it compares the RGB levels of the decoded image signal with the multiple detection patterns shown in Figure 9 (estimated levels of the output image signal), determines that the encoding range and decoding range of the closest (smallest difference) detection pattern are in line with the current standard, and determines whether the two ranges are consistent or inconsistent. Then, the process proceeds to step S211.

[0140] (Step S211) The detection unit 112 determines whether or not it has detected an inconsistency between the encoding and decoding standards. If the detection unit 112 determines that the encoding and decoding standards are consistent, that is, if it has not detected an inconsistency in the standards (Step S211: NO), it terminates the process as no correction is necessary. On the other hand, if the detection unit 112 determines that it has detected an inconsistency in the standards (Step S211: YES), it proceeds to the process in Step S213.

[0141] (Step S213) The detection unit 112 instructs the correction unit 113 to perform a correction along with the ID of the detection pattern that it determined to be inconsistent. Then, the process proceeds to step S215.

[0142] (Step S215) When the correction unit 113 receives a correction instruction from the detection unit 112, it determines the correction content associated with the ID of the detection pattern that was determined to be mismatched as the correction content to be applied to the image signal. Then, it terminates the process.

[0143] If the detection unit 112 detects a mismatch in the standards and determines the correction content, the correction unit 113 then applies the determined correction content to the image signal based on the imaging signal captured by the camera 50 in subsequent processing.

[0144] For example, the correction unit 113 acquires the decoded image signal output from the camera driver 110, performs correction processing on the acquired image signal, and then passes it to the application processing unit 111. As an example, the correction unit 113 may acquire the decoded image signal and perform correction processing using the MFT device, which is supported as an extension of Windows (registered trademark).

[0145] As described above, the information processing system 1 according to this embodiment includes a camera 50 (an example of an imaging device) and an information processing device 10 that acquires an image signal output from the camera 50 and performs processing to display a display image based on the acquired image signal on the display unit 15. The camera 50 comprises an imaging unit 51, an imaging processing unit 52, and a test signal generator 53 (an example of a test signal output unit). The imaging unit 51 outputs an imaging signal of a subject. The test signal generator 53 outputs a preset test image signal. The imaging processing unit 52 encodes the imaging signal output from the imaging unit 51 or the test image signal output from the test signal output unit 53 according to a first standard set in the camera 50, and outputs the encoded image signal. The information processing device 10 comprises a camera driver 110 (an example of an image processing unit), a detection unit 112, and a correction unit 113. The camera driver 110 decodes the image signal output from the camera 50 according to the second standard and outputs the decoded image signal. The detection unit 112 acquires the image signal after the camera driver 110 decodes the encoded image signal acquired from the camera 50, and detects an inconsistency between the first standard (encoding standard) and the second standard (decoding standard) based on the RGB levels (signal levels) of the acquired decoded image signal. If the detection unit 112 detects an inconsistency between the first standard and the second standard, the correction unit 113 determines a correction to correct the inconsistency between the first standard and the second standard based on the RGB levels of the decoded image signal acquired by the detection unit 112. The correction unit 113 also applies the determined correction to the encoded image signal of the imaging signal acquired from the camera 50 and performs a correction process.

[0146] As a result, the information processing system 1 detects and corrects inconsistencies between encoding and decoding standards based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information. This makes it possible to suppress the degradation of color reproduction in image signals caused by the coexistence of various standards in a simple manner.

[0147] For example, using a combination of multiple types of first standards (encoding standards) and multiple types of second standards (decoding standards), the levels of the image signals obtained when a test image signal is encoded using the first standard and then decoded using the second standard are pre-set as multiple detection patterns used by the detection unit 112 when detecting inconsistencies between the first and second standards (see Figures 8 and 9). The detection unit 112 then detects inconsistencies between the first and second standards by comparing the image signal obtained after encoding the test image signal acquired from the camera 50 with the image signal decoded by the camera driver 110 and the multiple detection patterns.

[0148] As a result, the information processing system 1 can detect standard inconsistencies without relying on standard information by comparing the RGB levels after decoding the image signal encoded by the camera 50 with the detection pattern for when the standards are inconsistent.

[0149] Furthermore, for example, for each detection pattern among multiple detection patterns in which the first standard (encoding standard) and the second standard (decoding standard) are inconsistent, the correction content for correcting the inconsistency between the first standard and the second standard is associated and set (see Figures 8 and 9). When the detection unit 112 detects an inconsistency between the first standard and the second standard, the correction unit 113 performs a correction process by applying the correction content associated with the corresponding detection pattern among the multiple detection patterns.

[0150] As a result, the information processing system 1 can pre-set correction content for each detection pattern when a standard mismatch occurs, and when a standard mismatch is detected, it can perform an appropriate correction corresponding to the detected mismatch pattern.

[0151] For example, the first standard (encoding standard) and the second standard (decoding standard) include color space standards, and the correction content includes conversion formulas for converting color spaces (see Figure 8).

[0152] As a result, the information processing system 1 can detect and correct any mismatch between the matrix coefficients of the encoded color space and the matrix coefficients of the decoded color space based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information.

[0153] For example, the first standard (encoding standard) and the second standard (decoding standard) include a range (number of grayscale levels), and the correction details include a conversion formula for converting the range (see Figure 9).

[0154] As a result, the information processing system 1 can detect and correct any mismatch between the encoding range and the decoding range based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information.

[0155] Furthermore, when the detection unit 112 detects an inconsistency between the first standard (encoding standard) and the second standard (decoding standard), it sends an instruction to the camera 50 to output a test image signal. When the imaging processing unit 52 of the camera 50 receives the instruction to output a test image signal, it encodes the test image signal output from the test signal generator 53 using the first standard and outputs the encoded image signal.

[0156] This allows the information processing system 1 to switch from the imaging signal to a test image signal and output it from the camera 50 when performing color reproduction compensation processing to detect and correct mismatches in standards.

[0157] Furthermore, the information processing device 10 according to this embodiment acquires an image signal output from a camera 50 (an example of an imaging device) and processes the display of a display image based on the acquired image signal on the display unit 15. The camera 50 selects either an imaging signal capturing a subject or a preset test image signal, encodes it according to a first standard set in the camera 50, and outputs the encoded image signal. The information processing device 10 also includes a camera driver 110 (an example of an image processing unit), a detection unit 112, and a correction unit 113. The camera driver 110 decodes the image signal output from the camera 50 according to a second standard and outputs the decoded image signal. The detection unit 112 instructs the camera 50 to output a test image signal, thereby acquiring the image signal after the camera driver 110 decodes the encoded image signal of the test image signal acquired from the camera 50, and detects an inconsistency between the first standard (encoding standard) and the second standard (decoding standard) based on the RGB level (signal level) of the acquired decoded image signal. If the detection unit 112 detects an inconsistency between the first standard and the second standard, the correction unit 113 determines a correction to correct the inconsistency between the first standard and the second standard based on the RGB levels of the decoded image signal acquired by the detection unit 112. The correction unit 113 also applies the determined correction to the encoded image signal acquired from the camera 50 and performs a correction process.

[0158] As a result, the information processing device 10 detects and corrects inconsistencies between encoding and decoding standards based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information. This makes it possible to suppress the degradation of color reproduction in image signals caused by the coexistence of various standards in a simple manner.

[0159] Furthermore, the control method in the information processing device 10 according to this embodiment is a control method in the information processing device 10 that acquires an image signal output from a camera 50 (an example of an imaging device) and performs the process of displaying a display image based on the acquired image signal on the display unit 15. The camera 50 selects either the imaging signal that captures the subject or a pre-set test image signal, encodes it according to a first standard set in the camera 50, and outputs the encoded image signal. The control method in the information processing device 10 includes the steps of: the camera driver 110 decodes the image signal output from the camera 50 according to the second standard and outputs the decoded image signal; the detection unit 112 instructs the camera 50 to output a test image signal, thereby acquiring the image signal after the camera driver 110 decodes the encoded image signal of the test image signal acquired from the camera 50, and detecting an inconsistency between the first standard (encoding standard) and the second standard (decoding standard) based on the RGB level (signal level) of the acquired decoded image signal; the correction unit 113, if the detection unit 112 has detected an inconsistency between the first standard and the second standard, determines a correction content to correct the inconsistency between the first standard and the second standard based on the RGB level of the decoded image signal acquired by the detection unit 112; and the correction unit 113 performs a correction process by applying the determined correction content to the encoded image signal of the imaging signal acquired from the camera 50.

[0160] As a result, the control method in the information processing device 10 detects and corrects inconsistencies between encoding and decoding standards based on the RGB levels after decoding the image signal encoded by the camera 50, without using standard information. This makes it possible to suppress the degradation of color reproduction of the image signal caused by the coexistence of various standards in a simple manner.

[0161] <Second Embodiment> Next, a second embodiment of the present invention will be described. In the first embodiment, a configuration in which the camera is built into the information processing device was described, but in this embodiment, a configuration in which the camera is an external device (peripheral device) connected to the information processing device will be described.

[0162] Figure 12 is a configuration diagram showing an example of an information processing system 1A according to this embodiment. As shown in Figure 12, the information processing system 1A comprises an information processing device 10A and a camera 50A. The basic hardware configuration of the information processing device 10A is the same as the hardware configuration of the information processing device 10 shown in Figure 2, but with the camera 50 removed. Note that the information processing device 10A may also include a camera 50, and may be connected to the camera 50A.

[0163] The information processing device 10A is connected to the camera 50A by wire or wireless connection and can acquire images captured by the camera 50A and display them on the display unit 15. For example, the information processing device 10A and the camera 50A are connected by a USB cable 5 that conforms to the USB Type-C standard. For example, the USB cable 5 used to connect to the camera 50A is connected to a USB connector 16 (see Figure 2).

[0164] Thus, even in an information processing system 1A where the camera 50A is connected to the information processing device 10A as an external camera, the color reproduction compensation processing described in the first embodiment can be applied.

[0165] Although the example shown involves connecting the camera 50A and the information processing device 10A with a USB cable 5, other connection methods are also acceptable.

[0166] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to those described above, and various design changes can be made without departing from the spirit of this invention.

[0167] In the above embodiment, an example using a 75% color bar as the test image signal was described, but the invention is not limited to this. For example, an image containing at least one color from each color of the 75% color bar may be used as the test image signal.

[0168] Furthermore, although the above embodiment describes an example in which the image signal acquired from the camera 50 is decoded by the camera driver 110, the location of decoding is not limited to the camera driver 110. For example, the application processing unit 111 may perform decoding using a standard set in the application. Alternatively, a display driver (not shown) that controls the display unit 15 may perform the decoding.

[0169] Furthermore, in the above embodiment, the detection unit 112 was described using screen capture from the display unit 15 as an example of a method for acquiring the decoded image signal in order to detect the mismatch in standards, but it is not limited to this. For example, when decoding is performed by the camera driver 110 as in the above embodiment, the detection unit 112 may acquire the decoded image signal output from the camera driver 110 and detect the mismatch in standards.

[0170] Furthermore, in the above embodiment, two standards, "ITU-R BT.601," which is the standard for older television broadcasting (SDTV), and "ITU-R BT.709," which is the standard for HDTV, were used as examples to explain the encoding and decoding standards of image signals, but other standards may also be used. Also, even if three or more standards are mixed, the number of combinations of encoding standards and decoding standards will increase, but the above embodiment can be applied by setting the respective detection patterns. As an example of other standards, for example, "ITU-R BT.2020," which is the standard for 4K / 8K UHD (Ultra High Definition) broadcasting, may also be included.

[0171] Furthermore, the information processing device 10 described above has a computer system inside. The processing in each configuration of the information processing device 10 described above may be performed by recording a program for realizing the functions of each configuration of the information processing device 10 onto a computer-readable recording medium, loading the program recorded on this recording medium into the computer system, and executing it. Here, "loading the program recorded on the recording medium into the computer system and executing it" includes installing the program into the computer system. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer system" may include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. Thus, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM.

[0172] Furthermore, the recording medium also includes internal or external recording media accessible from the distribution server for distributing the program. The program may be divided into multiple parts, downloaded at different times, and then combined in each configuration of the information processing device 10. The distribution servers for each of the divided programs may also be different. Moreover, "computer-readable recording media" includes volatile memory (RAM) within computer systems that act as servers or clients when a program is transmitted over a network, which retains the program for a certain period of time. The program itself may also be intended to implement some of the functions described above. Furthermore, the program may be a so-called differential file (differential program) that can implement the functions described above in combination with a program already recorded in the computer system.

[0173] Furthermore, some or all of the functions of the information processing device 10 in the above-described embodiment may be implemented as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually processorized, or some or all of them may be integrated into a single processor. In addition, the method of implementing the integrated circuit is not limited to LSIs; it may also be implemented using dedicated circuits or general-purpose processors. Furthermore, if an integrated circuit technology that can replace LSIs emerges due to advances in semiconductor technology, an integrated circuit using that technology may be used.

[0174] Furthermore, although the above-described embodiment described an example in which the information processing device 10 is a clamshell-type (notebook-type) PC, it may also be a desktop-type or tablet-type PC. [Explanation of Symbols]

[0175] 1 Information processing system, 5 USB cable, 10 Information processing device, 15 Display unit, 16 USB connector, 17 Communication unit, 18 Storage unit, 19 Input unit, 20 EC, 21 Power supply unit, 22 Battery, 50 Camera, 51 Imaging unit, 52 Imaging processing unit, 53 Test signal generator, 100 System processing unit, 101 CPU, 102 GPU, 103 Memory controller, 104 I / O controller, 105 System memory, 110 Camera driver, 111 Application processing unit, 112 Detection unit, 113 Correction unit

Claims

1. An information processing system comprising an imaging device and an information processing device that acquires an image signal output from the imaging device and performs processing to display an image based on the acquired image signal on a display unit, The imaging device is An imaging unit that outputs an imaging signal of the subject, A test signal output unit that outputs a pre-set test image signal, An imaging processing unit encodes the imaging signal output from the imaging unit or the test image signal output from the test signal output unit according to a first standard set in the imaging device, and outputs the encoded image signal. Equipped with, The aforementioned information processing device is An image processing unit that decodes the image signal output from the aforementioned imaging device according to the second standard and outputs the decoded image signal, A detection unit acquires an image signal obtained by the image processing unit after decoding the encoded image signal of the test image signal acquired from the imaging device, and detects a mismatch between the first standard and the second standard based on the signal level of the acquired decoded image signal. If the detection unit detects an inconsistency between the first standard and the second standard, the correction unit determines a correction content to correct the inconsistency between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit. Equipped with, The correction unit, A correction process is performed by applying the correction content to the image signal after encoding the imaging signal acquired from the imaging device. Information processing system.

2. Each of the levels of the image signal obtained when the test image signal is encoded using the first standard and then decoded using the second standard, using a combination of multiple types of the first standard and multiple types of the second standard, is pre-set as one of several detection patterns used by the detection unit when detecting inconsistencies between the first standard and the second standard. The detection unit is The image processing unit compares the image signal obtained by decoding the encoded test image signal acquired from the imaging device with a plurality of detection patterns to detect inconsistencies between the first standard and the second standard. The information processing system according to claim 1.

3. For each of the detection patterns among the multiple detection patterns in which the first standard and the second standard are inconsistent, the respective correction content for correcting the inconsistency between the first standard and the second standard is associated and set. The correction unit, If the detection unit detects an inconsistency between the first standard and the second standard, the correction process is performed by applying the correction content associated with the corresponding detection pattern from among the multiple detection patterns. The information processing system according to claim 2.

4. The first and second standards include color space standards, The aforementioned correction includes a conversion formula for converting color spaces. The information processing system according to claim 1.

5. The first and second standards mentioned above include standards for the number of grayscale levels. The aforementioned correction includes a conversion formula for converting the number of grayscale levels. The information processing system according to claim 1.

6. The detection unit is When detecting a mismatch between the first standard and the second standard, an instruction to output the test image signal is transmitted to the imaging device. The aforementioned imaging processing unit, Upon receiving an instruction to output the test image signal, the test image signal output from the test signal output unit is encoded according to the first standard, and the encoded image signal is output. The information processing system according to claim 1.

7. An information processing device that selects either an imaging signal capturing a subject or a pre-set test image signal, encodes it according to a first standard set in the imaging device, and outputs an image signal after encoding, and acquires the image signal output from the imaging device, and performs a process of displaying a display image based on the acquired image signal on a display unit, An image processing unit that decodes the image signal output from the aforementioned imaging device according to the second standard and outputs the decoded image signal, A detection unit that, by instructing the imaging device to output the test image signal, obtains an image signal obtained from the imaging device after the image processing unit decodes the encoded image signal of the test image signal, and detects a mismatch between the first standard and the second standard based on the signal level of the decoded image signal obtained, If the detection unit detects an inconsistency between the first standard and the second standard, the correction unit determines a correction content to correct the inconsistency between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit. Equipped with, The correction unit, A correction process is performed by applying the correction content to the image signal after encoding the imaging signal acquired from the imaging device. Information processing device.

8. A control method for an information processing device that selects either an imaging signal capturing an object or a pre-set test image signal, encodes it according to a first standard set in the imaging device, and outputs an image signal after encoding, acquires the image signal output from the imaging device, and performs a process of displaying a display image based on the acquired image signal on a display unit, wherein the device The image processing unit decodes the image signal output from the imaging device according to the second standard and outputs the decoded image signal, The detection unit instructs the imaging device to output the test image signal, thereby acquiring the image signal obtained from the imaging device after the image processing unit decodes the encoded image signal of the test image signal, and detecting the mismatch between the first standard and the second standard based on the signal level of the acquired decoded image signal. If the correction unit detects an inconsistency between the first standard and the second standard by the detection unit, the correction unit determines a correction content to correct the inconsistency between the first standard and the second standard based on the signal level of the decoded image signal acquired by the detection unit. The correction unit performs a correction process by applying the correction content to the image signal obtained from the imaging device after encoding, A control method including

Images